Search Tweets¶
This project is inspired and led by "How to analyze the sentiment of your own Tweets" by Jessica Garson!
In this project, we pulls the Tweets from a certain Twitter account from the past 7 days and gives you a score to let you know exactly how his/her week has been.
In [1]:
import requests
import pandas as pd
import json
import ast
import yaml
# Before you can connect the Twitter API,
# you’ll need to set up the URL to ensure it has the right fields so you get the right data back.
# you can get the particular endpoint on https://developer.twitter.com/en/docs/twitter-api/search-overview
# we are using Recent Search Endpoint for the tweets from specific account on the past 7 days
def create_twitter_url(handle): # the parameter handle here is the account username
max_results = 100
mrf = "max_results={}".format(max_results)
q = "query=from:{}".format(handle)
url = "https://api.twitter.com/2/tweets/search/recent?tweet.fields=lang&{}&{}".format(mrf, q)
return url
# To access the configuration file you created while setting up config.yaml
def process_yaml():
with open("config.yaml") as file:
return yaml.safe_load(file)
# To access the bearer token from your config.yaml file
def create_bearer_token(data):
return data["search_tweets_v2"]["bearer_token"]
# To format the headers to pass in your bearer_token and url
def twitter_auth_and_connect(bearer_token, url):
headers = {"Authorization": "Bearer {}".format(bearer_token)}
response = requests.request("GET", url, headers=headers)
if response.status_code != 200:
raise Exception(response.status_code, response.text)
return response.json()
def no_tweets(res_json):
if res_json == {"meta": {"result_count": 0}}:
print("The Twitter handle entered hasn't Tweeted in 7 days.")
def connect_to_azure(data):
# you can find the endpoint url in your azure project
azure_url = "https://senti-on-tweet.cognitiveservices.azure.com/"
sentiment_url = "{}text/analytics/v2.1/sentiment".format(azure_url)
# the subscription key we are using here is the API key
subscription_key = data["azure"]["subscription_key"]
return sentiment_url, subscription_key
def azure_header(subscription_key):
return {"Ocp-Apim-Subscription-Key": subscription_key}
def create_document_format(res_json):
data_only = res_json["data"]
doc_start = '"documents": {}'.format(data_only)
str_json = "{" + doc_start + "}"
dump_doc = json.dumps(str_json)
doc = json.loads(dump_doc)
return ast.literal_eval(doc)
def sentiment_scores(headers, sentiment_url, document_format):
response = requests.post(sentiment_url, headers=headers, json=document_format)
return response.json()
def mean_score(sentiments):
sentiment_df = pd.DataFrame(sentiments["documents"])
return sentiment_df["score"].mean()
def week_logic(week_score):
if week_score > 0.75 or week_score == 0.75:
print("This account had a positive week")
elif week_score > 0.45 or week_score == 0.45:
print("This account had a neautral week")
else:
print("This account had a negative week, I hope it gets better")
# select the paragraph, CTRL + / to (un)comment out the function(s)
# This is an testing function for providing example results to help you understand the result
def analysis(handle):
url = create_twitter_url(handle)
data = process_yaml()
bearer_token = create_bearer_token(data)
res_json = twitter_auth_and_connect(bearer_token, url)
no_tweets(res_json)
sentiment_url, subscription_key = connect_to_azure(data)
headers = azure_header(subscription_key)
document_format = create_document_format(res_json)
sentiments = sentiment_scores(headers, sentiment_url, document_format)
week_score = mean_score(sentiments)
data = pd.DataFrame.from_dict(res_json['data'])
print("Base on the ", data.shape[0], "tweets this account post from past 7 days, we get a score of ", week_score)
week_logic(week_score)
return data
# def main():
# handle = input("Enter Account Username: ")
# url = create_twitter_url(handle)
# data = process_yaml()
# bearer_token = create_bearer_token(data)
# res_json = twitter_auth_and_connect(bearer_token, url)
# no_tweets(res_json)
# sentiment_url, subscription_key = connect_to_azure(data)
# headers = azure_header(subscription_key)
# document_format = create_document_format(res_json)
# sentiments = sentiment_scores(headers, sentiment_url, document_format)
# week_score = mean_score(sentiments)
# print("Base on the ", pd.DataFrame.from_dict(res_json['data']).shape[0], "tweets this account post from past 7 days, we get a score of ", week_score)
# week_logic(week_score)
# if __name__ == "__main__":
# main()
There are some examples for you to understand what the result you be
In [ ]:
main()
If you want to digging more about the company, you can uncomment the analysis function and comment the main function.
In [3]:
# settings for the pandas expression
pd.set_option('display.max_colwidth', None)
PUMA Sentiment Analysis¶
In [4]:
puma = analysis("PUMA")
puma
Base on the 25 tweets this account post from past 7 days, we get a score of 0.6810827732086182 This account had a neautral week
Out[4]:
| id | lang | text | |
|---|---|---|---|
| 0 | 1428060567428812800 | und | @EmlynBegley 🤝 |
| 1 | 1427747052474281993 | en | RT @jgault13: This is awesome. What began as a $4,761 donation has now ballooned to eight times that amount thanks to a number of generous… |
| 2 | 1427746996023144459 | en | RT @fast_women: Nope, that wasn't the last update. @puma has agreed to match the donation, so Molly Seidel's run at the @FalmouthRR raised… |
| 3 | 1427746979661107205 | en | RT @fast_women: Olympic medalist Molly Seidel served as the starter at today's Falmouth Road Race, and then began the race in last place. F… |
| 4 | 1427746676358488064 | en | Last weekend, @ByGollyMolly12 raised $19,044 for @TommysPlace_ by passing 4,761 runners after starting at the very back of the @FalmouthRR. One week after THAT RACE, too 🤯 We're so proud of you Molly – we’ll match the donation to support such an amazing cause. #PUMAFam https://t.co/CMFDxTOffR |
| 5 | 1427568600454508548 | und | RT @therealshammgod: 8/21 https://t.co/qfTVLPjzNZ |
| 6 | 1427292438356545536 | en | 🖤🤍 @dualipa in the new Suede Mayu 🖤🤍 https://t.co/dQ1ScbqmxY |
| 7 | 1427276317259706377 | en | Greatest. Of. All. Time. 🐐 @usainbolt https://t.co/DVxafLzGeB |
| 8 | 1427187289709481984 | en | RT @DUALIPA: The latest from @PUMA . New Mayu out tomorrow 🖤 shot by Mario Sorrenti #ad https://t.co/70dLcMYUdH |
| 9 | 1426983973922693120 | en | RT @erinasimon: Glad you like our @PUMA RKDO gift package @sjokz!\n\nAppreciate you and thank you for being an awesome esports rep 🔥😃🙌🏽 https… |
| 10 | 1426950822924111880 | und | @LILCOBEY 🤝 #PUMAFam |
| 11 | 1426950461198938113 | und | #SuedeSunday @MELOD1P 🛸💕 https://t.co/9MmkxeWkqx |
| 12 | 1426948450361483272 | en | RT @Oratile011: Top 5 Greatest silhouette ever. |
| 13 | 1426901382976741377 | en | Beautiful from every angle 🧡 #SuedeSunday\n📷: formstripes (IG) https://t.co/YaMCoX7sBL |
| 14 | 1426892392347676672 | en | RT @HotFreestyle: Nipsey Hussle would’ve been 36 years old today, Happy Birthday & Rest In Peace 🙏🏽🕊🏁 https://t.co/grD03FIVng |
| 15 | 1426892327377833985 | en | RT @Genius: reminder from nipsey: you're supposed to be here. #verified https://t.co/KOvUz4ZPzI |
| 16 | 1426578192219914242 | en | RT @Kgudie_: Okay my baby just got here 😭😭🤍🤍 https://t.co/VFWZ5qhgp6 |
| 17 | 1426552524199366659 | en | @iLOVEnewyork83 @andreagrimes Worth it 💚 |
| 18 | 1426543456038658048 | en | Marble Suede lowkey fire. Michaelangelo where you at? https://t.co/hwonVWM0vD |
| 19 | 1426483722627559424 | en | RT @WSeriesRacing: 🔍 a woman's place is 𝐢𝐧 𝐦𝐨𝐭𝐨𝐫𝐬𝐩𝐨𝐫𝐭 |
| 20 | 1426185376809558017 | ru | PUMA Wild Rider в действии. Движение и бесконечная энергия города в твоих Wild Rider 🤸♂️ |
| 21 | 1426084341331988481 | ru | Задай темп этой игре, оставив соперника ни с чем с ULTRA 1.3 и FUTURE Z 1.2 💥 |
| 22 | 1426083687586832387 | ru | PUMA Faster Football – это свобода передвижения на поле, чтобы твоя игра была максимально динамичной ⚡ |
| 23 | 1425873053259534339 | en | PUMA FAM BRINGING HOME THOSE MEDALS. #OnlySeeGreat https://t.co/dlj3EkUkpf |
| 24 | 1425819369456640002 | en | 🖤🤍 @dannapaola in Mayze\nhttps://t.co/MsygWDlljW https://t.co/lk2ufCCVOv |
In [4]:
text_puma = puma[puma['lang']=='en'].drop(['id', 'lang'], axis=1).drop_duplicates('text')
text_puma.count()
text_puma
Out[4]:
| text | |
|---|---|
| 0 | 🖤🤍 @dualipa in the new Suede Mayu 🖤🤍 https://t.co/dQ1ScbqmxY |
| 1 | Greatest. Of. All. Time. 🐐 @usainbolt https://t.co/DVxafLzGeB |
| 2 | RT @DUALIPA: The latest from @PUMA . New Mayu out tomorrow 🖤 shot by Mario Sorrenti #ad https://t.co/70dLcMYUdH |
| 3 | RT @erinasimon: Glad you like our @PUMA RKDO gift package @sjokz!\n\nAppreciate you and thank you for being an awesome esports rep 🔥😃🙌🏽 https… |
| 6 | RT @Oratile011: Top 5 Greatest silhouette ever. |
| 7 | Beautiful from every angle 🧡 #SuedeSunday\n📷: formstripes (IG) https://t.co/YaMCoX7sBL |
| 8 | RT @HotFreestyle: Nipsey Hussle would’ve been 36 years old today, Happy Birthday & Rest In Peace 🙏🏽🕊🏁 https://t.co/grD03FIVng |
| 9 | RT @Genius: reminder from nipsey: you're supposed to be here. #verified https://t.co/KOvUz4ZPzI |
| 10 | RT @Kgudie_: Okay my baby just got here 😭😭🤍🤍 https://t.co/VFWZ5qhgp6 |
| 11 | @iLOVEnewyork83 @andreagrimes Worth it 💚 |
| 12 | Marble Suede lowkey fire. Michaelangelo where you at? https://t.co/hwonVWM0vD |
| 13 | RT @WSeriesRacing: 🔍 a woman's place is 𝐢𝐧 𝐦𝐨𝐭𝐨𝐫𝐬𝐩𝐨𝐫𝐭 |
| 17 | PUMA FAM BRINGING HOME THOSE MEDALS. #OnlySeeGreat https://t.co/dlj3EkUkpf |
| 18 | 🖤🤍 @dannapaola in Mayze\nhttps://t.co/MsygWDlljW https://t.co/lk2ufCCVOv |
| 19 | RT @Bratz: Thursday! 👄👟 @PUMA #bratz https://t.co/vdGDnOExSH |
| 20 | RT @Babyyhairz: The Hussle Way \n“Miami Story” August 13\n@PUMA x @themarathonclothing https://t.co/zuWagsy0Yk |
| 22 | RT @brkicks: First look at LaMelo Ball’s first signature shoe with Puma called the MB1 😮 @NickDePaula https://t.co/AKzCYxdh6l |
| 23 | RT @NickDePaula: The Puma Jet — the brand’s private plane for its athletes — is still one of the best endorsement perks out. \n\nHere’s how L… |
In [5]:
retweets_puma = text_puma[text_puma['text'].str.contains('RT')]
retweets_puma
Out[5]:
| text | |
|---|---|
| 2 | RT @DUALIPA: The latest from @PUMA . New Mayu out tomorrow 🖤 shot by Mario Sorrenti #ad https://t.co/70dLcMYUdH |
| 3 | RT @erinasimon: Glad you like our @PUMA RKDO gift package @sjokz!\n\nAppreciate you and thank you for being an awesome esports rep 🔥😃🙌🏽 https… |
| 6 | RT @Oratile011: Top 5 Greatest silhouette ever. |
| 8 | RT @HotFreestyle: Nipsey Hussle would’ve been 36 years old today, Happy Birthday & Rest In Peace 🙏🏽🕊🏁 https://t.co/grD03FIVng |
| 9 | RT @Genius: reminder from nipsey: you're supposed to be here. #verified https://t.co/KOvUz4ZPzI |
| 10 | RT @Kgudie_: Okay my baby just got here 😭😭🤍🤍 https://t.co/VFWZ5qhgp6 |
| 13 | RT @WSeriesRacing: 🔍 a woman's place is 𝐢𝐧 𝐦𝐨𝐭𝐨𝐫𝐬𝐩𝐨𝐫𝐭 |
| 19 | RT @Bratz: Thursday! 👄👟 @PUMA #bratz https://t.co/vdGDnOExSH |
| 20 | RT @Babyyhairz: The Hussle Way \n“Miami Story” August 13\n@PUMA x @themarathonclothing https://t.co/zuWagsy0Yk |
| 22 | RT @brkicks: First look at LaMelo Ball’s first signature shoe with Puma called the MB1 😮 @NickDePaula https://t.co/AKzCYxdh6l |
| 23 | RT @NickDePaula: The Puma Jet — the brand’s private plane for its athletes — is still one of the best endorsement perks out. \n\nHere’s how L… |
PATAGONIA Sentiment Analysis¶
In [4]:
# Pull out all the tweets for Patagonia in last 7 days
patagonia = analysis("patagonia")
patagonia
Base on the 63 tweets this account post from past 7 days, we get a score of 0.7208683774584815 This account had a neautral week
Out[4]:
| id | lang | text | |
|---|---|---|---|
| 0 | 1428479629480890379 | en | Join Patagonia grantees @CAUSE805 and @CFROG_vc for a community meeting on August 21st to stop the expansion of a toxic natural gas compressor station in Ventura. |
| 1 | 1428427233568251905 | en | @Nirwin_Images Sorry to hear you're disappointed by the current hat selection and color preferences. We'll let our designers know, and hopefully you'll find one you like in the future! |
| 2 | 1428348017568927750 | en | @gianluto Hello, we are incredibly sorry to hear about the issues you are having. Would you please DM us with your order number and email so we can look into this for you? https://t.co/BQQTdfLPtb |
| 3 | 1428128460237463553 | en | Run to: A film series about runners finding activism through sport.\n\nWatch the first episode: Corriendo Para Salvar Una Cuenca | Run To Save a Watershed: https://t.co/MBwrfmKySF https://t.co/j0uU8VHza0 |
| 4 | 1428118862973739009 | en | @attamusk You'll find our Back For Good wolf hat here: https://t.co/6eTieEDiTy |
| ... | ... | ... | ... |
| 58 | 1425961595214069762 | en | Cozy sweatshirts made with soft, Regenerative Organic Certified™ Pilot Cotton. Built to last for seasons to come. |
| 59 | 1425961542969790464 | en | Cozy sweatshirts made with soft, Regenerative Organic Certified™ Pilot Cotton. Built to last for seasons to come. |
| 60 | 1425961483087749122 | en | Cozy sweatshirts made with soft, Regenerative Organic Certified™ Pilot Cotton. Built to last for seasons to come. |
| 61 | 1425961399704985606 | en | Cozy sweatshirts made with soft, Regenerative Organic Certified™ Pilot Cotton. Built to last for seasons to come. |
| 62 | 1425961325335781376 | en | Cozy sweatshirts made with soft, Regenerative Organic Certified™ Pilot Cotton. Built to last for seasons to come. |
63 rows × 3 columns
In [5]:
text_patagonia = patagonia.drop(['id', 'lang'], axis=1).drop_duplicates('text')
text_patagonia.count()
Out[5]:
text 52 dtype: int64
In [6]:
Social_Good = text_patagonia[text_patagonia['text'].str.contains('protect|help|environment')]
print(Social_Good.count())
Social_Good
text 9 dtype: int64
Out[6]:
| text | |
|---|---|
| 11 | Patagonia grantee @waterfirstngo collaborates with Indigenous communities in Canada to address critical water challenges. Click to learn how they are creating solutions to sustain access to clean water and how you can help support their work. |
| 12 | Patagonia grantee @waterfirstngo collaborates with Indigenous communities in Canada to address critical water challenges. Click to learn how they are creating solutions to sustain access to clean water and ways you can help support their work. |
| 18 | Join Patagonia grantee @Savannainst and their partners on August 21st for a field day at the Memorial 4H Camp Demonstration Farm. Learn about agroforestry techniques for building soil health, protecting water quality, enhancing wildlife habitat and diversifying profits. |
| 19 | Take action with Patagonia @TheNationsRiver to help ensure a resilient and just future for communities across the nation. Call on federal leaders to invest in clean water, green jobs and nature-based infrastructure. |
| 24 | Take action with Patagonia grantee @CalWild as they work to ensure that 30% of California's lands and waters are protected by 2030. |
| 39 | Join Patagonia grantee @UM_Waterkeeper in calling on Montana leaders to protect the state's cold water fisheries for future generations. Click to add your voice. |
| 44 | @MariannaFila HI Marianna, we use OnTrac because it helps meet our customers' needs and it is more sustainable to use local shipping options per region. If you currently having issues with one of your orders delivering with OnTrac, DM us so we can help fix the situation! https://t.co/BQQTdfLPtb |
| 45 | Is donating clothes actually helpful? Many shelters in your town will take clothing donations. But they don't want to be burdened with donations that aren't clean or useful. Comics journalist Sarah Mirk learns how to donate with dignity: https://t.co/kj4VoNEqwR https://t.co/WaOMSj0kX1 |
| 49 | @MackFordFan98 Perfect, we are incredibly happy to hear that Mack! If you have any questions or concerns during the exchange process, please let us know and we will be more than happy to help. 😊 |
In [11]:
retweets_patagonia = text_patagonia[text_patagonia['text'].str.contains('RT')]
retweets_patagonia
Out[11]:
| text | |
|---|---|
| 3 | RT @conservationall: Did you miss last week's Conservation Alliance Breakfast? You can still watch or share the recorded event. Tune in now… |
| 35 | RT @FightFossils: Biden could cancel Formosa’s permit with the stroke of a pen. \n\n@POTUS, save our lives from the industry that's poisoning… |
In [8]:
reply = text_patagonia[text_patagonia['text'].str.contains('Hi|Thank you|Hey|Sorry|sorry')]
print(reply.count())
reply
text 15 dtype: int64
Out[8]:
| text | |
|---|---|
| 1 | @Nirwin_Images Sorry to hear you're disappointed by the current hat selection and color preferences. We'll let our designers know, and hopefully you'll find one you like in the future! |
| 2 | @gianluto Hello, we are incredibly sorry to hear about the issues you are having. Would you please DM us with your order number and email so we can look into this for you? https://t.co/BQQTdfLPtb |
| 10 | @mrBriskly Hey Phil, thanks for reaching out. We have a filter on our website that allows you to search for specific materials, you can select cotton or hemp for example. We have the Men's Long-Sleeved Work Henley Pocket Tee which is a hemp/cotton blend. DM us for more suggestions! https://t.co/BQQTdfLPtb |
| 28 | @sarahjeanefink We're sorry we don't currently have what you're looking for. Each season we increase the number of styles with additional size range. Please reach out to us in a DM with more specifics that we can pass along to our product designers. We'd love to hear from you! https://t.co/BQQTdfLPtb |
| 30 | Sam doesn’t consider himself a great baker or a cyclist. So why did he join his friends to start Bread Bike, a bicycle-delivery breadmaking business in San Luis Obispo? His reasons are deliciously unexpected. https://t.co/XnIhbXlM2E https://t.co/CAjgwJSPkI |
| 33 | @Ryan32198411 Thank you so much for your feedback and suggestions Ryan, we truly appreciate it! We will ensure this gets passed along to our Product Design Team so they are aware there is a need/want for these hats. |
| 34 | @insanitynow5 Hey Billy! The Interstate Trucker Hats are pretty rad. Items are sometimes reintroduced back into our line and customer feedback is one of the contributing factors towards such decisions. Your comments have been passed along your comments to our Design Team. |
| 37 | @ThomasKHuddles2 We're so sorry to see this error! Please reach out to us in a DM so that we can get you fixed up right away. https://t.co/BQQTdfLPtb |
| 41 | @dezeerae Hi Des, we're sorry to hear about the disappointment with sizing. We know that sizing has not been inclusive and we're working each season to expand our product sizing to outfit a wider range of body types. You can see what we have available here, https://t.co/Vbs9sIQApT. |
| 42 | @ckmcdkc Hey Courtney! It's a super rad hoody, right?! That is our new Regenerative Organic Cotton Essential Hoody. It is expected to launch online in early September so check out https://t.co/qXNsy9NEuR around then. |
| 43 | @ThatVatoJules Hi Julio, we're so sorry to hear you haven't had a great experience with our website. Please DM us so we can further assist with these issues. https://t.co/BQQTdfLPtb |
| 47 | @djbuffnstuff Hi Craig, we're so sorry to hear about trouble with your order arriving. Please get back to us privately so we can further assist you. https://t.co/BQQTdfLPtb |
| 48 | @JKBAD Hey, thanks for bringing this to our attention. Unfortunately, it is a scam. We'll DM you with next steps. |
| 50 | @MackFordFan98 Hello Mark, we are incredibly sorry to hear your zipper is no longer working. Would you please DM us with more details so we can get this resolved for you? https://t.co/BQQTdfLPtb |
| 51 | @tdersmom Hello! We are incredibly sorry to hear about your zipper. Would you please send us a DM with more information so we can go over our options to get your zipper repaired? https://t.co/BQQTdfLPtb |
JanSport Sentiment Analysis¶
In [11]:
# Pull out all the tweets for JanSport in last 7 days
JanSport= analysis("JanSport")
JanSport
The Twitter handle entered hasn't Tweeted in 7 days.
--------------------------------------------------------------------------- KeyError Traceback (most recent call last) <ipython-input-11-e864d2b88562> in <module> 1 # Pull out all the tweets for JanSport in last 7 days ----> 2 JanSport= analysis("JanSport") 3 JanSport <ipython-input-1-5fbff253cd5d> in analysis(handle) 83 sentiment_url, subscription_key = connect_to_azure(data) 84 headers = azure_header(subscription_key) ---> 85 document_format = create_document_format(res_json) 86 sentiments = sentiment_scores(headers, sentiment_url, document_format) 87 week_score = mean_score(sentiments) <ipython-input-1-5fbff253cd5d> in create_document_format(res_json) 49 50 def create_document_format(res_json): ---> 51 data_only = res_json["data"] 52 doc_start = '"documents": {}'.format(data_only) 53 str_json = "{" + doc_start + "}" KeyError: 'data'
Columbia Sentiment Analysis¶
In [12]:
columbia = analysis("Columbia1938")
columbia
Base on the 2 tweets this account post from past 7 days, we get a score of 0.9218319654464722 This account had a positve week
Out[12]:
| id | lang | text | |
|---|---|---|---|
| 0 | 1426283619644026891 | en | This 106-mile course doles out 32,940 feet of elevation gain in a series of brutal climbs, although each one is set against a stunning panoramic backdrop. \n\nFind out what makes the @UTMBMontBlanc so badass:\nhttps://t.co/Y3eorH2wFU |
| 1 | 1425215856708132870 | en | To help you make the right choice, we’ve listed seven of the most important qualities to keep in mind when you’re shopping for a backpack. \nhttps://t.co/P1ByHRq3Fu |
In [ ]: