Have you ever wanted to have a visualisation or dataframe accessible from your laptop or phone without having to run the code every time? Wouldn’t it be just great if you could leave it running in the background and have a web address in which you could access the Data anytime, anyplace (with an internet connection)? Especially one that could automatically update when new data was available.
Turns out, you can, and it’s not that difficult at all. I taught myself how to do it over the last week just from Google, but there’s a lot of rubbish out there so here’s the easiest way that I’ve found. There might be better alternatives that I don’t know about, so please let me know if there is any. I’m genuinely curious. What you can definitely guarantee from this method, however, is simplicity. It also won’t cost you a penny. I’m also absolutely not a web developer so I apologise in advance if I murdered any terminology.
This article doesn’t expect much from you, a basic understanding of Python will do. A little knowledge of HTML would be beneficial, but absolutely not essential. Understanding Flask already would make this a piece of cake, but if you know Flask you probably know how to do this already. The structure will look a little bit like this and you could genuinely have this up and going within the hour, no problem.
- Setting up flask
- Basic routing
- Pandas Dataframes
- Matplotlib plots
- Hosting on PythonAnywhere
Setting up Flask
This isn’t an article about web design, as I’m certainly not qualified to talk about that. This is an article, simply about how to get dataframes and plots on your website. Most importantly plots and dataframes that automatically update. You could just ‘save.fig’ your plots and shove them in your static folder and constantly update them when fresh data comes in, but that’s drudgery and I don’t like drudgery.
So, first of all, what we’re going to do is have a very simple website with two pages. One page showcases your scintillating graphs and one page highlights your beautiful Pandas dataframe.
It will look exactly like this.
Very ugly, I know, but you can make it prettier with a bit of CSS or JS, but that’s for another time. This is simply to show you how to get them in there, you can worry about making them look pretty later.
Here’s the code. I will go through it section by section. As you can see there is only 69 lines in the main file.
We can start with lines 0–16. This is all stuff that needs importing, the first chunk (line 4) are flask packages that help you handle all the routing and the website structure. They are all pretty self-explanatory in name and you don’t have to think about these too much. Some of them, as you can see, you wont actually need for the basic version of this. They are extremely useful, however, so I tend to always import them.
The next chunk (lines 6–12) of imports are all for plotting your data. Again these are all pretty common and self explanatory. The only one that may stand out to you is line 11 – ‘matplotlib.use(‘Agg)’. For some reason that I don’t fully understand, the default plt backend does not work well with Flask – this line of code fixes it. If you don’t use plt, don’t worry, but if you do I suggest you include it. Line 15 is just another package that we will need later down the line which I’ll do my best to explain when we come to it.
Now, we need to get some data to use. I have two approaches in this file, the ones that aren’t commented out is what I feel is the easiest way to get your data in. Just have Pandas bring it in through a CSV file. However, there are a couple of issues with this. Firstly, When hosting the website on PythonAnywhere (PA) later on, the path may be different and relative to your PA environment, not your local desktop. So make sure that you specify the correct path depending on whether you are testing locally or are trying to publish it online. Secondly, this will not update automatically unless you automatically update the CSV files. If you want to have data that automatically update then my personal preference is to define a function that scrapes it from the internet and cleans it into a Pandas dataframe. I won’t go into scraping in this article but an example of how to do it is in this GitHub repo along with the rest of the code involved in this small project. So, to make the plots ‘dynamic’, all you have to do is find a dynamic source of data (the internet is usually best) and create a file that scrapes for it every time the main file runs. Lines 19 and 21 call functions in another file which scrapes the internet to do exactly this.
Right, finally, on to the flask app itself. The first thing you have to do is simply initialise the app –line 25. There is room for personalisation – but I wouldn’t. It’s easy to mess things up down the line. I always stick to that exact line.
Basic routing
Flask uses route decorators to let you specify which functions should be executed at each URL. Therefore, the idea here is to create two URLs that return two different functions – one that shows us our dataframe and one that shows us our plot. The basic route decorator can be seen in line 28. This decorators argument is simply " / " which effectively tells Flask that this is our homepage and the function below should run from our starting URL – in my case: http://jasher4994.pythonanywhere.com/. As you can see, if you remove the " / " from the URL then nothing changes. Therefore, to create new pages with extended URLs you can simply create a new one with the route decorator. This is what we will do below.
Pandas dataframes
So, firstly, we can do one for our Pandas Dataframe which we can also make our homepage. We’ve already seen the route operator in action in line 28, so we can extend it and create a new page called ‘Pandas’ in line 29. This time we do the exact same thing apart from creating a new URL called ‘Pandas’ and state that we want to use the methods "POST" and "GET". These are the two most common HTTP methods and are used to send and request data to a server respectively.
Therefore, we now have two URLs that will call the same function. The function is defined below on line 30. It’s a traditional Python function which returns another function – "render_template()". This function is extremely useful and effectively tells Flask to render in an HTML template from your templates folder (the template has to be in a subdirectory called "templates" so make sure it is). Below is the HTML template that you will need to create. It’s dead simple, only lines 7–10 are all you need to add to the traditional HTML template. Line 7 doesn’t even need to be there it’s just a heading tag.
So, what’s going on? In line 33 of our main file, we pass ‘table’ as an argument to our render template function. The table is just our pandas dataframe in which the ‘to_html()’ method has been applied. I’m pretty sure you can guess what that does. The rest of the line is just specifying whether we want things like an index and the column titles. Then, within our HTML file, we can use this table.
Flask actually uses Jinja2 (a python template engine to create HTML) here which is what you can see in lines 7–10. You don’t have to worry about this too much, but these 3 lines effectively display your table in HTML.
It’s as easy as that! If you run the main file in your terminal now you should get a local link that you can copy and paste into your browser, and you should see your dataframe!
Matplotlib plots
Matplotlib plots seemed a bit more difficult at first to work within Flask but in the end, it turns out they are quite simple. We will work our way from bottom to top in this section, so bear with me.
If your graph never changes, you can just serve it as a static file in your images folder and reference it directly.
However, if it does need to change regularly then this approach is for you. For example, the data I use here is on football teams expected clean sheets (the probability they will not concede a goal). This, of course, changes game by game depending on the opponent. So, I have another file that regularly scrapes and updates this data. If I were to serve this as a static file, it would not get updated and would just display the first weeks’ graph. With this method, the plot will be displayed on the fly and I will never need to update it. I’ve never tested this on "real-time" data and suspect it wouldn’t work very well and would probably require some JavaScript. If your data updates are daily, or even hourly, I suspect this method will work absolutely fine.
Referring back to the Matplotlib section (line 50–63) first in the Python file, we can first look at the create_fig() function. All this does is create a figure(graph) and plot the data we want it to. In this case, it plots ‘team’ on the x-axis, and this teams ‘expected goals’ on the y. I’ve changed the colour and the rotation of the x-ticks just for aesthetic reasons.
Moving up, we get to the plot_png() function (44–48). This one is a little bit more complicated but not much. In short, it creates a webpage that shows only the graph we displayed earlier – in png format. ‘BytesIO()’ creates an in-memory buffer, optionally filled with the data you provide as an argument, and lets you do file-like operations on it.
This will produce a webpage with solely the image that we created on it. Again, creating a new route as before can be seen here. That’s useful, but not useful enough. Surely, in the future, we might want some text describing our lovely plot or maybe some other plots on the same page – not just a page full of one big plot.
We, therefore, want our plot to be part of the page, not the full page.
Therefore, we create one final route (lines 38–40) and render our second and final template, which is below.
As can be seen, it’s dead easy. Just two lines of code different from the standard HTML template – and one of them is a heading! Line 7 is the important one here, it goes to the route we created earlier with the full page png, takes the image, and then places it onto our HTML file. Now the image is part of our page and doesn’t swallow up the whole thing.
Ta-Da! You now have a way to show your plots and dataframes on a browser. Only locally, however, so this isn’t much use. Luckily it’s extremely simple to post them online for the world to see with pythonanywhere. The following section will show you just how to do so.
Hosting on Python anywhere
Now, for the final step, hosting your website. pythonanywhere (PA) is a great, simple place to do this. If done correctly, you can be up and running in less than five minutes too. So, once you have your three files (your main Flask file and your two HTML files) create a PA account, you’ll only need a beginner account for this – which is free. Once, you’re all signed up you should see a dashboard that looks like this.
We will mostly be using the "Web" and "Files" tab here, but it doesn’t take a wild imagination to think about what the other tabs offer.
Firstly, we need to create our web app. So on the dashboard, on the left, click "Add a new web app" and then follow the instructions. Select whichever Python version you have used and of course, Flask is your framework. It will then create a web app for you, you can specify the name – I called mine ‘mysite’ to keep it simple.
So now what we need to do is, get our files into PA. If you know Git, fantastic, PA has a console that you can open and pull your repository directly and then you’re basically ready to go.
For people who don’t know Git, it’s also super easy and I’ll show you below how to do it. Firstly, go to the files section. Then, on the left, click on your web app path (mysite). In this directory add in your Flask File. Then within this directory, create a new directory called "templates". It’s important for Flask that you do this step exactly, so it knows where to look for the HTML templates. Then within this new templates directory add in your two HTML files. It should look like this.
Almost there! One final thing you have to check is to make sure that the source code is directly specified. If it doesn’t look like this..
Then change it so it does.
..and finally! Hit that big sexy green button at the top to reload your app and then click the link.
Et voila, you now have Pandas and Matplotlib running on a webpage. Python anywhere will run this endlessly for you for free, but you’ll have to log in every three months to let them know you’re still alive and you still want to run the website. They will email you a week before they do, however, so no need to set any reminders.
Once you develop a nice skeleton like this, it’s incredibly simple to make a website look reasonably professional these days with resources like https://getbootstrap.com/ kicking around.
I really hope this helps and any feedback would be greatly appreciated. Some similar articles I’ve written that you might like are below.
If I've inspired you to join medium I would be really grateful if you did it through this link - it will help to support me to write better content in the future.If you want to learn more about Data Science, become a certified data scientist, or land a job in data science, then checkout 365 data science through my affiliate link.Cheers,
James
