Body

How to build a simple web-scraping script to monitor gym discounts in Windows 10 with gmail…

I used Python 3.6.1 on Windows 10 with the following 3 packages:

  • beautifulsoup4 for parsing html pages
  • requests for downloading webpages
  • smtplib for sending emails

If you do not have these 3 packages, you can run the following 3 commands:

pip install beautifulsoup4
pip install requests
pip install smtplib

The first step to scrap the webpage is to download the webpage, and then use beautifulsoup to parse the webpage.

import bs4, requests, smtplib
r = requests.get('https://somewebsite.com/some_offers')
soup = bs4.BeautifulSoup(r.text, 'html.parser')

Then we can examine the body of the parsed html page to find the pricing information. I found it at the 20th entry in body tag’s children by using .contents . People can just examine the web page by right clicking to choose view source and hitting CTRL-F to find the price.

print(soup.body.contents[19])
<div class="span4 pricing-table">
<ul>
<li class="price">
<span class="currency-symbol">$</span> <strong>29</strong>
<sup>00</sup> <em>/ down</em><br/>
<em style="display:none">+0.00 for second member</em>
<em style="display:none">+0.00 for each additional member</em>
</li>
<li class="price" style="display:block">
<span class="currency-symbol">$</span> <strong>14</strong> <sup>99</sup>
<em>/ month</em><br/>
<em style="display:none">+0.00 for second member</em>
<em style="display:none">+0.00 for each additional member</em>
</li>
</ul>

Since the price is enclosed by the <strong> and <sup> tags, I can extract the price by using soup.find_all('strong') to find all the <strong> tags. The same can be done with the <sup> tags.

strong = soup.find_all('strong')
initiation = strong[0].contents[0] + '.' + sup[0].contents[0]
sup = soup.find_all('sup')
monthly = strong[1].contents[0] + '.' + sup[1].contents[0]
print('Initiation Fees = ', initiation) 
# The output is `Initiation Fees = 29.00`
print('Monthly Fees = ', monthly)
# The output is `Monthly Fees = 14.99`

In order to use gmail to send us the email, we need an App password. You can click on this Link to the Help Center to see how to do so.

Assume that you’ve followed the steps, you should get to the following page to get your App password:

After we got our App password (mine is a 16-character password like: abcdabcdabcdabcd ), we can use the following codes to send us an email.

title = 'Subject: Fitness Price: Initiation Fee {} Monthly Fee {}'.format(initiation, monthly)
appkey = 'abcdabcdabcdabcd'
from_address = 'your_email@gmail.com'
to_address = ['your_email@gmail.com','other_email@gmail.com']
conn = smtplib.SMTP('smtp.gmail.com', 587) 
conn.ehlo() # Start the connection
conn.starttls() # Start TLS encryption. Password will be encrypted.
conn.login(from_address, appkey)
conn.sendmail(from_address, to_address, title)
conn.quit()

After running the above codes, we should see an email in our inbox:

If you only want to be notified when the price is good, then you can place an if statement to only send email if the monthly fees (and/or the initiation fees) are below a certain price. Finally, the complete code (with some try-except handling just in case parsing or anything weird happens) is shown below:

Now, we can set up a scheduled task in Windows 10. First we hit Windows Key + R to open run and key in taskschd.msc to open the Task Scheduler. Right click on Task Scheduler Library to Create Task

Fill in the name Fitness_Alert and click on Triggers Tab.

Pick a time to trigger this job. For me, I picked 9AM every morning. Click OK to confirm. Then click on the Actions Tab.

Fill in the location of the python executables in Program/script field and the script we’ve created (fitness_alert.py) in the arguments. Click ok to confirm.

If you do not know where is your Python executable file, you can use the following code snippets to find out.

import os, sys
os.path.dirname(sys.executable)

# Mine is at 'C:\Users\kungh\Anaconda3'

Finally, we can test the script by right clicking to choose Run .

We should see an email in the inbox. Done!!

Now I just need to wait for the deals to show up again…..LOL.


Source link
Tags
Back to top button
close
Thanks !

Thanks for sharing this, you are awesome !

Pin It on Pinterest

Share This

Share this post with your friends!