Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Page says “Uses Twitter's unofficial API (no rate limits or developer account required)”. What is this unofficial API?


It means using the "guest token" (gt) for sending GraphQL queries. The gt is public and is distributed via the twitter.com public web page. The gt is the same for every member of the public.

To see the gt, read the contents of twitter.com

For example, using Chrome

    chrome view-source:mobile.twitter.com  
    Ctrl-F gt=
or curl

    curl https://mobile.twitter.com|grep -o gt=[0-9]*
To retrieve tweets three steps are required: 1. retrieve the "guest token" (gt), 2. retrieve the "REST ID" (rest_id) for the twitter.com user and 3. submit a GraphQL query to retrieve the user's tweets.

Javascript provided by Twitter in the twitter.com can do these three steps automatically (but this requires using a Javascript-enabled browser), or it can be done without a browser, e.g., with a different scripting language (personally I use the shell).

For example, some folks use Python:

https://pypi.org/project/twint/


Wow!

But there must be some kind of IP based rate limiting or something, right? At least for DDoS protection even if it's not intended to prevent scrapping.


I would guess that it's some kind of reverse-engineered internal API that official apps use.


Never have I been more happy to be wrong. It used to be impacted by rate limiting quite a bit, but it looks like the situation has improved, probably due to this unofficial API.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: