For many companies, it’s interesting to analyse tweets where people mention your company’s handle. This data can be used to identify feedback topics and questions that customers regularly ask the company. When we understand the key topics, we can then put an action plan in place to prevent problems from happening in the first place.
Tweepy is a package that allows easier interaction with the Twitter API. In this article I will walk through how to set yourself up on the Twitter developer platform, and use Tweepy to grab any tweets where your handle is mentioned.
First of all, Twitter publishes definitions of different terms and that’s important to understand as it dictates how we call the API.
Definitions (Twitter, 2022)
Tweets: A message posted to Twitter containing text, photos, a GIF, and/or video.
Mentions: A Tweet containing another account’s Twitter username, preceded by the "@" symbol. For example: "Hello @TwitterSupport!"
Replies: A reply is when you respond to another person’s Tweet.
We want mentions because we want to find out what our customers say about us.
Getting set up on Twitter developer
The majority of the challenge is getting set up on the Twitter developer platform and authenticating yourself. After this – it’s quite straightforward.
Go to the twitter developer portal: https://developer.twitter.com/en and sign up.
You’ll need to fill in some basic detail and validate your email.
Then you’ll set up a project
After you have a project, Twitter will provide you with the API_key and API_key_secret. These are both critical so make sure you save them.
Next you need to set up elevated access. This is on your project page.
They ask you to confirm some basic personal details, and then you have to answer why you want enhanced access.
Finally, you have to agree to the terms and conditions and then your application will be reviewed. Mine was instant but I have heard that it can take a few days.
Now you have got elevated access, you need to enable authentication to your app.
Your app needs to have read and write permission, and I have selected native app.
Twitter requests a callback URI and a website URL – these can seem problematic for analysis you are running locally. For the callback URI you can use: http://127.0.0.1/return and use any website for the website URL.
Then we need to get the last two set of credentials we need; access token and access token secret.
Go to key and tokens within your app and generate the access tokens and save them locally.
There are four items we need to authenticate ourself with:
- api_key
- api_key_secret
- access_token
- access_token_secret
We now have all four items so we can start using Tweepy to gather our mentions.
Using Tweepy to connect to the Twitter API
First, make sure that the tweepy package is installed.
Then import the package:
Next we need to authenticate ourselves using the tokens we generated earlier. I have made dummy keys to avoid giving you all access to my twitter account.
We initialise the authentication object using the following code and then set the access token. Finally we initialise the API using the authentication object.
To get tweets where our handle is mentioned, we need to use ‘mentions_timeline’ from the API. The ‘count’ argument determines how many tweets are returned.
The output of this is a JSON file. We are only interested in the text of the tweet so we loop over the JSON object containing the data, and only print out only the text element.
The full code is in my github.
Conclusion
The Twitter API is a brilliant resource for natural language processessing (NLP) datasets, however calling it directly can be challenging. Tweepy provides a package that makes it easier to grab the data that you need, so you can get on with generating insights for the business. In this example, I wanted to grab mentions, but Tweepy allows you to get your own tweets, the tweets of a specific user, or even control your account by following / blocking users. The full list of functionality can be found in the documentation.
As an alternative, if your company has licenses for Sprinklr (which is a well known social media management platform), this provides a nice front end to access the Twitter API without requiring any code.
References
Twitter (2022) About different types of Tweets. Available at: https://help.twitter.com/en/using-twitter/types-of-tweets (Accessed: 26 October 2022).
I like to write about Data Science for business users and I’m passionate about using data to deliver tangible business benefits.
You can connect with me on LinkedIn and follow me on Medium to stay up to date with my latest articles.