Python: Iwe afọwọkọ A Google Autosuggest Extract of Trends for Your Keyiche Search Keywords
Gbogbo eniyan fẹran Awọn aṣa Google, ṣugbọn o jẹ ẹtan diẹ nigbati o ba de si Awọn Koko-ọrọ Tail Long. Gbogbo wa fẹran aṣoju naa iṣẹ awọn aṣa google fun gbigba awọn oye lori ihuwasi wiwa. Sibẹsibẹ, awọn nkan meji ṣe idiwọ ọpọlọpọ lati lo fun iṣẹ to lagbara;
- Nigbati o ba nilo lati wa titun koko ọrọ, Ní bẹ ko to data lori Awọn aṣa Google
- Aisi API osise fun ṣiṣe awọn ibeere si awọn aṣa google: Nigba ti a ba lo awọn modulu bii pytrends, lẹhinna a ni lati lo awọn olupin aṣoju, tabi a ni idina.
Ninu nkan yii, Emi yoo pin Iwe-akọọlẹ Python ti a ti kọ si okeere awọn ọrọ-ọrọ aṣa nipasẹ Google Autosuggest.
Gba ati tọju Awọn abajade Idojukọ Aigbagbe Ju Aago
Ṣebi a ni awọn ọrọ-ọrọ 1,000 irugbin lati firanṣẹ si Google Autosuggest. Ni ipadabọ, a le rii to 200,000 gan-an awọn ọrọ-ọrọ. Lẹhinna, a nilo lati ṣe kanna ni ọsẹ kan lẹhinna ki o ṣe afiwe awọn iwe data wọnyi lati dahun awọn ibeere meji:
- Awọn ibeere wo ni titun koko akawe si akoko ikẹhin? Eyi ṣee ṣe ọran ti a nilo. Google ro pe awọn ibeere wọnyẹn ti di pataki siwaju sii - nipa ṣiṣe bẹ, a le ṣẹda ojutu ti ara ẹni Google Autosuggest wa!
- Awọn ibeere wo ni awọn koko-ọrọ ko si mọ aṣa?
Iwe afọwọkọ naa rọrun pupọ, ati pupọ julọ koodu ti Mo pin Nibi. Koodu ti a ti ni imudojuiwọn ṣafipamọ data lati awọn iṣiṣẹ ti o kọja ati afiwe awọn aba lori akoko. A yago fun awọn apoti isura data ti o da lori faili bi SQLite lati jẹ ki o rọrun - nitorinaa gbogbo ibi ipamọ data nlo awọn faili CSV ni isalẹ. Eyi n jẹ ki o gbe faili wọle ni Tayo ati ṣawari awọn aṣa ọrọ koko fun iṣowo rẹ.
Lati Lo akosile Python yii
- Tẹ koko ọrọ irugbin rẹ ti o yẹ ki o firanṣẹ si adaṣe adaṣe: awọn ọrọ-ọrọ .csv
- Ṣatunṣe awọn eto mimọ fun iwulo rẹ:
- LANGUAGE: aiyipada “en”
- Orilẹ-ede: aiyipada “awa”
- Ṣeto iwe afọwọkọ lati ṣiṣẹ lẹẹkan ni ọsẹ kan. O tun le ṣiṣẹ pẹlu ọwọ bi o ṣe fẹ.
- Lo keyword_suggestions.csv fun itupalẹ siwaju:
- akọkọ_riran: eyi ni ọjọ ti ibeere naa farahan fun igba akọkọ ninu adaṣe-aye
- kẹhin_riran: ọjọ ti a ti rii ibeere naa fun akoko ikẹhin
- is_tuntun: ti akọkọ_seen == last_seen a ṣeto eyi si otitọ - Kan ṣe àlẹmọ lori iye yii lati gba awọn iwadii aṣa tuntun ni aifọwọyi Google.
Eyi ni Koodu Python
# Pemavor.com Autocomplete Trends
# Author: Stefan Neefischer (stefan.neefischer@gmail.com)
import concurrent.futures
from datetime import date
from datetime import datetime
import pandas as pd
import itertools
import requests
import string
import json
import time
charList = " " + string.ascii_lowercase + string.digits
def makeGoogleRequest(query):
# If you make requests too quickly, you may be blocked by google
time.sleep(WAIT_TIME)
URL="http://suggestqueries.google.com/complete/search"
PARAMS = {"client":"opera",
"hl":LANGUAGE,
"q":query,
"gl":COUNTRY}
response = requests.get(URL, params=PARAMS)
if response.status_code == 200:
try:
suggestedSearches = json.loads(response.content.decode('utf-8'))[1]
except:
suggestedSearches = json.loads(response.content.decode('latin-1'))[1]
return suggestedSearches
else:
return "ERR"
def getGoogleSuggests(keyword):
# err_count1 = 0
queryList = [keyword + " " + char for char in charList]
suggestions = []
for query in queryList:
suggestion = makeGoogleRequest(query)
if suggestion != 'ERR':
suggestions.append(suggestion)
# Remove empty suggestions
suggestions = set(itertools.chain(*suggestions))
if "" in suggestions:
suggestions.remove("")
return suggestions
def autocomplete(csv_fileName):
dateTimeObj = datetime.now().date()
#read your csv file that contain keywords that you want to send to google autocomplete
df = pd.read_csv(csv_fileName)
keywords = df.iloc[:,0].tolist()
resultList = []
with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
futuresGoogle = {executor.submit(getGoogleSuggests, keyword): keyword for keyword in keywords}
for future in concurrent.futures.as_completed(futuresGoogle):
key = futuresGoogle[future]
for suggestion in future.result():
resultList.append([key, suggestion])
# Convert the results to a dataframe
suggestion_new = pd.DataFrame(resultList, columns=['Keyword','Suggestion'])
del resultList
#if we have old results read them
try:
suggestion_df=pd.read_csv("keyword_suggestions.csv")
except:
suggestion_df=pd.DataFrame(columns=['first_seen','last_seen','Keyword','Suggestion'])
suggestionCommon_list=[]
suggestionNew_list=[]
for keyword in suggestion_new["Keyword"].unique():
new_df=suggestion_new[suggestion_new["Keyword"]==keyword]
old_df=suggestion_df[suggestion_df["Keyword"]==keyword]
newSuggestion=set(new_df["Suggestion"].to_list())
oldSuggestion=set(old_df["Suggestion"].to_list())
commonSuggestion=list(newSuggestion & oldSuggestion)
new_Suggestion=list(newSuggestion - oldSuggestion)
for suggest in commonSuggestion:
suggestionCommon_list.append([dateTimeObj,keyword,suggest])
for suggest in new_Suggestion:
suggestionNew_list.append([dateTimeObj,dateTimeObj,keyword,suggest])
#new keywords
newSuggestion_df = pd.DataFrame(suggestionNew_list, columns=['first_seen','last_seen','Keyword','Suggestion'])
#shared keywords with date update
commonSuggestion_df = pd.DataFrame(suggestionCommon_list, columns=['last_seen','Keyword','Suggestion'])
merge=pd.merge(suggestion_df, commonSuggestion_df, left_on=["Suggestion"], right_on=["Suggestion"], how='left')
merge = merge.rename(columns={'last_seen_y': 'last_seen',"Keyword_x":"Keyword"})
merge["last_seen"].fillna(merge["last_seen_x"], inplace=True)
del merge["last_seen_x"]
del merge["Keyword_y"]
#merge old results with new results
frames = [merge, newSuggestion_df]
keywords_df = pd.concat(frames, ignore_index=True, sort=False)
# Save dataframe as a CSV file
keywords_df['first_seen'] = pd.to_datetime(keywords_df['first_seen'])
keywords_df = keywords_df.sort_values(by=['first_seen','Keyword'], ascending=[False,False])
keywords_df['first_seen']= pd.to_datetime(keywords_df['first_seen'])
keywords_df['last_seen']= pd.to_datetime(keywords_df['last_seen'])
keywords_df['is_new'] = (keywords_df['first_seen']== keywords_df['last_seen'])
keywords_df=keywords_df[['first_seen','last_seen','Keyword','Suggestion','is_new']]
keywords_df.to_csv('keyword_suggestions.csv', index=False)
# If you use more than 50 seed keywords you should slow down your requests - otherwise google is blocking the script
# If you have thousands of seed keywords use e.g. WAIT_TIME = 1 and MAX_WORKERS = 5
WAIT_TIME = 0.2
MAX_WORKERS = 20
# set the autocomplete language
LANGUAGE = "en"
# set the autocomplete country code - DE, US, TR, GR, etc..
COUNTRY="US"
# Keyword_seed csv file name. One column csv file.
#csv_fileName="keyword_seeds.csv"
CSV_FILE_NAME="keywords.csv"
autocomplete(CSV_FILE_NAME)
#The result will save in keyword_suggestions.csv csv file