Platforms, Python Practice Projects, and Picking up Flask: A blog story

Warning: raw language used. I have a filthy mouth sometimes. Don't read if you don't like seeing adult language used. I am raw and unfiltered here just like I am in real life!

Note to anyone interested in metrics: sh*t appears 7 times, f*ck appears twice.

I'm primarily a coder. I have been paid to dev stuff like an iOS app. I have also been paid to fix bugs in big, corporate iOS applications. I've further been paid to write Python scripts and perform pentests on iOS, Android, and Web applications.

There's a world of difference between my skillset and the skillsets of "actual developers".

Actual devs build things. I'm just good at code. I've built things, but not on the regular; nor with the knowledge of actual devs so much as a Computer Science graduate with no strong direction and simply trying to get back onto his financial feet so he can move the fuck out of his grandma's home and live a much happier life.

I tend to pick up what is needed and/or asked of me. I've become aware of the wide range of tools and roles that a "dev" can occupy. There is a lot out there, and I'll be damned if I am expected to learn garbage toolchains like "React.js" that are completely antithetical to my entire approach of keeping shit minimal.

Recently, I applied to get onto a certain website's platform, and after a casual chat with an engineer, I was asked to build a "small" (to me, it was not small at the time) backend project. You're given some REST endpoints, details on how it should work (though, not enough details), and some sample input/output to drive things. You're given hints about object design. It is up to you to decide how to build it.

So, because I love Python, and wanted to keep things "flexible", I opted to learn something called "Flask".

Below is the largely disorganized personal notes from the project. This is actually how I work when building and learning things. It will come off as a bit "scatterbrained" because this is how I reflect on myself while working. I kinda "chat" back-and-forth with myself via a markdown blog. I don't really "talk" a whole lot to myself, except through text and handwriting.

For this project, I went in blog heavy.

I have changed the names of REST endpoints and removed URLs that hint at the project. I think it would be very "wrong" to expect me NOT to share my research and newly-learned tools and skills with the rest of the world, so I can't be expected of that, so I have done due diligence to hide any "meaningful" details.

I look forward to making future blog posts and to growing my skills as a Python user...programmer, developer, and hacker, and to building things for others so that I can get paid and move into a better environment so that I can be happy and content...well, more so :)

xxxxx - Development blog

Refresh JWT POST /some-token-endpoint/refresh ----- User login POST /some-token-endpoint ----- Log out current user DELETE /some-token-endpoint ----- Get current user's info GET /myinfo ----- Create idea POST /getideas ----- Delete idea DELETE /getideas/:id ----- Get a page of ideas (1 page = 10 ideas) GET /getideas ----- Update idea PUT /getideas/:id ----- User Signup POST /getusers

This seems easy enough, just getting started is the hard part.

We can use flask to define each of the API endpoints.

We don't list it in the API, but there is implied a login page at / .

Current day and time: Sat Feb 23 2019 22:55

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 12m30s

Total time spent: 12m30s

On each new update/entry to this blog,

we will prepend with the current day/time,

delivery date, and current and total time spent

Useful links for this project

deleted to protect project

deleted to protect project

https://jwt.io/introduction/

https://tools.ietf.org/html/rfc7519

Current day and time: Mon Feb 25 2019 01:33

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 1hr20m

Total time spent: 1hr32m

Initial pages created.



/ /user_registration /getusers /some-token-endpoint

Todo soon:

Handle user registration database initial creation

database update Handle user login Handle user logout

Continuing...

Somehow, my flask isn't linked up to Python anymore, and I have too many versions installed, so I am going to take time to clean up my Python installs by removing the existing ones, and then installing from scratch via brew.

Current day and time: Mon Feb 25 2019 23:46

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 44m01s

Total time spent: 1hr44m33m

I fixed the Flask setup...ate up a bit of time...Python3 as well.

I moved some of the code around, but haven't made much progress.

I need to do some research on how to handle user registration and how to store the data in the database.

We will spend time on that now.

So, a good idea is templates.

We should do that now.

Ok, we have managed to transition over to a template page system smoothly and it feels natural to create and manage pages this way.

We still need to create the initial user table and decide on how we are going to store ideas for each user.



user table: user_id - email - password_hash

The user_id could link to other table content so that we could stash ideas possibly in a blob object or something?

I am not sure what the "best" way to do this is, so I will have to look into that.

Current day and time: Tue Feb 26 2019 00:14

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 24m08s

Total time spent: 2hr08m41m

Installed flask-wtf .

Moved stuff around to be more 'proper'.

App structure:



xxxxx/ config.py xxxxx.py run.sh app/ __init__.py routes.py templates/ index.html user_registration.html users.html

Everything still builds and runs fine.

We still need to do user registration, but we are following the tutorial.

Gotta read before we dive in.

Time check.

Current day and time: Thu Feb 28 2019 02:21

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 28m11s

Total time spent: 2hr36m52m

Went a little deeper into building forms.

Actual LoginForm() object.

Current day and time: Thu Feb 28 2019 02:48

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 25m47s

Total time spent: 3hr01m39m

We've progressed a little bit but are now on implementing error messages when the user enters invalid form data.

LoginForm() and RegistrationForm() .

Taking a break to sleep.

Current day and time: Thu Feb 28 2019 03:28

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 39m12s

Total time spent: 3hr40m51m

My xxxxx: ______________________________ usermode: 10 trial: 10 continental: 10 averaje: 10 OK Cancel

I FINALLY progressed through the database section of the tutorial:

I created a User and Idea 'model' which is basically the table definition.

To fix the db when we change shit:



flask db migrate flask db upgrade

When you make changes, you can comment



flask db migrate -m "your comment"

Now that we've done that we are ready to start working with logins:

We have been working hard, and so we should take a break.

Current day and time: Thu Feb 28 2019 22:06

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 1hr42m57s

Total time spent: 4hr23m48s

Current diff:

fixed a bunch of shit as im working through the login tutorial, something broke but it turned out to be these two imports causing errors on the next line down in another file when I try to import something from __init__.py .

Current day and time: Fri Mar 1 2019 02:46

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 1hr29m23s

Total time spent: 5hr53m11s

We still need to check out the shell and login from that level.

Ugh shit is broken :(

YES

WE HAVE LIFTOFF!!!

I have tested the login form and it is working as expected.

This is kind of exciting again!

We can take a break. Need to clear head and prepare for next steps.

Current day and time: Fri Mar 1 2019 03:29

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 40m48s

Total time spent: 5hr04m36s

Next steps:

User registration easy

Implement the ideas thingy Add an idea Delete an idea Display ideas



Alright

Time to grind this shit out!

User Registration!

We have it working.

You can create a new user immediately and login with them immediately.

The idea list is all that remains.

Let us checkpoint and continue.

Current day and time: Fri Mar 1 2019 23:36

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 22m48s

Total time spent: 5hr28m04s

Implement the ideas thingy Add an idea Delete an idea Display ideas



On a successful user registration, we should take the user to the "My Ideas" page.

I'm thinking I only actually need to implement the endpoints and not the pages themselves -_- whoops!

Ok...

Lets re-arrange stuff.

I'm doing user-registration first.

If I can test this in curl...first I need to fix curl...brew curl...

Ok...

Progress report:

I've got POST /getusers reflecting our input data.

We need it to compute a jwt token and a refresh_token and return it as json.



{ "jwt" : "...", "refresh_token" : "..." }

Looks like we can use Flask-JWT for that.

https://pythonhosted.org/Flask-JWT/

I've got it installed and kind of rigged into routes.py but I am erroring when attempting to authenticate at /auth ...



./run.sh * Serving Flask app "xxxxx.py" * Environment: production WARNING: Do not use the development server in a production environment. Use a production WSGI server instead. * Debug mode: off * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) [2019-03-02 02:56:46,265] ERROR in app: Exception on /auth [POST] Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise raise value File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python3.7/site-packages/flask_jwt/__init__.py", line 115, in _default_auth_request_handler username = data.get(current_app.config.get('JWT_AUTH_USERNAME_KEY'), None) AttributeError: 'NoneType' object has no attribute 'get' 127.0.0.1 - - [02/Mar/2019 02:56:46] "POST /auth HTTP/1.1" 500 -

This looks like a possible bug in Flask_JWT but I cannot be 100% sure here...

Because this whole thing is taking me so long, I've asked them for another 4 days to work on it. If I grind hard, I can make it in 4 days.

Thing is, this part is representing a roadblock at the moment...

Even if I were to backtrack/start-from-scratch, re-build everything in bare-minimum Flask, when I get to this part, I will still have to solve how to generate the JWT...

Flask_JWT was supposed to have solved this for me, but even using it is a little challenging...

I am taking a lot of new information in, and this requires some mental background processing on my part.

So, to recap:



/getusers is reflecting our json post data now /auth is crashing for Flask_JWT

Current day and time: Sat Mar 2 2019 03:03

Delivery date: Sat Mar 2 2019 06:35 AM

Current time spent: 1hr50m01s

Total time spent: 7hr18m05s

Wednesday, March 6th, 2019

Updating stats:

Current day and time: Wed Mar 6 2019 14:29

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr0m0s

Total time spent: 7hr18m05s

We have been granted the extension.

Check this out:

We need to restart this from scratch, we overbuilt.



User Signup: POST /getusers Refresh JWT: POST /some-token-endpoint/refresh User login: POST /some-token-endpoint Log out: DELETE /some-token-endpoint Get current user's info: GET /myinfo Create idea: POST /getideas Delete idea: DELETE /getideas/:id Get a page of ideas (1 page = 10 ideas): GET /getideas Update idea: PUT /getideas/:id

For each method available:

User signup Refresh JWT User login User logout Get current user's info Create an idea Delete an idea Get a page of ideas Update an idea

We will need to re-create the database.



class User: name email password_hash class Idea: user_id text usermode trial continental averaje UserTable IdeaTable

First commit. Hello world.

Current day and time: Wed Mar 6 2019 14:49

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr8m12s

Total time spent: 7hr26m17s

We'll need to create the database and tables again next.

Done.

Current day and time: Wed Mar 6 2019 15:19

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr23m49s

Total time spent: 7hr50m06s

Next up, actually implementing each functionality.

We should logically start with user registration.

User registration

Parse out the inputs Verify inputs Pass verified-inputs to new User object commit User to db Construct JWT and return required tokens

Verifying inputs.

Current day and time: Wed Mar 6 2019 23:05

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr28m0s

Total time spent: 8hr18m06s

First I heard of flask_jwt .

Then I heard of flask_jwt_extended .

Then I heard of flask_jwt_auth

All of these tutorials are massive pains!

OH SHIT

IT FUCKING WORKS

WE ARE GENERATING JWTs!!!

The magic sauce is



retval = { 'access_token' : create_access_token(identity=u.id), 'refresh_token' : create_refresh_token(identity=u.id) }

We are actually getting both a JWT and a refresh_token back.

My main question is still how the refresh_token works in the first place, that isn't explained in the JWT intro, but I'm still expected to know what that is and how it works when I submit it...

Oh right, there's some decorator like @refresh_token_required or something.

This comes from the library flask_jwt_extended so thank you for being my savior here.

We still have all of the other APIs to implement, but registerUser is officially DONE!

For each method available:

1. User signup - DONE

Refresh JWT User login User logout Get current user's info Create an idea Delete an idea Get a page of ideas Update an idea

Current day and time: Thu Mar 7 2019 23:30

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 1hr31m02s

Total time spent: 9hr49m08s

Note to self:

The reason we had so much trouble getting SECRET_KEY in there was that we configure our app from an object defined, called Config. To solve the problem, we had to move SECRET_KEY and JWT_SECRET_KEY into the class definition.

Lets do login.

That should be easy.



Mar 8 1:57am: TIL Flask routes expect to return Responses or strings, tuples, etc, but not Dicts!

Looks like login is done.

We could spend a few minutes to clean up debugging messages.

1. User signup - DONE

2. User login - DONE

Refresh JWT User logout Get current user's info Create an idea Delete an idea Get a page of ideas Update an idea

Current day and time: Thu Mar 8 2019 02:13

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr55m11s

Total time spent: 10hr44m19s

Refresh token:



POST /some-token-endpoint/refresh

{'jwt': '', 'refresh_token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1NTIwMzA5OTcsIm5iZiI6MTU1MjAzMDk5NywianRpIjoiYjBjOGNhNDMtZmQzYi00OWI5LWFjM2YtZmU4ZWU0YjljY2UyIiwiZXhwIjoxNTUyMDMxNTk3LCJpZGVudGl0eSI6MSwidHlwZSI6InJlZnJlc2gifQ.9Akh0uY4ou8DBarkp2lDqS38dlIebqnwk0ss3J5B05s'}

JWT="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1NTIwMzA5OTcsIm5iZiI6MTU1MjAzMDk5NywianRpIjoiMWQ5NmU4MGQtOWQyZi00Mjg2LWEyMzItNGYxNDY5NjI5MGYyIiwiZXhwIjoxNTUyMDMxODk3LCJpZGVudGl0eSI6MSwiZnJlc2giOmZhbHNlLCJ0eXBlIjoiYWNjZXNzIn0.HyMCR5mFdcoUi7VUjA3LeiTDGn8Ccsy08Y9Bbc0LL9M" REFRESH_TOKEN="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1NTIwMzA5OTcsIm5iZiI6MTU1MjAzMDk5NywianRpIjoiYjBjOGNhNDMtZmQzYi00OWI5LWFjM2YtZmU4ZWU0YjljY2UyIiwiZXhwIjoxNTUyMDMxNTk3LCJpZGVudGl0eSI6MSwidHlwZSI6InJlZnJlc2gifQ.9Akh0uY4ou8DBarkp2lDqS38dlIebqnwk0ss3J5B05s" ./curl_refresh.sh "$JWT" "$REFRESH_TOKEN"

Looks like we are successfully returning new JWTs!!!

Note to self to save selves from future trouble:



JWT_HEADER_NAME JWT_HEADER_TYPE

Lots of Config options in flask-jwt-extended

1. User signup - DONE

2. User login - DONE

3. Refresh JWT

User logout Get current user's info Create an idea Delete an idea Get a page of ideas Update an idea

Current day and time: Fri Mar 9 2019 02:51

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr35m23s

Total time spent: 11hr19m42s

Logout

Seems like we did it.

1. User signup - DONE

2. User login - DONE

3. Refresh JWT

4. User logout

Get current user's info Create an idea Delete an idea Get a page of ideas Update an idea

Current day and time: Fri Mar 8 2019 04:56

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr20m08s

Total time spent: 11hr39m50s

Get current user's info

This was super-easy.

We still have to do something about that gravatar url thing.

We can come back to that.

1. User signup - DONE

2. User login - DONE

3. Refresh JWT

4. User logout

5. Get current user's info

Create an idea Delete an idea Get a page of ideas Update an idea

Current day and time: Fri Mar 8 2019 05:10

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr10m44s

Total time spent: 11hr50m34s

Create an idea

So far so good...

1. User signup - DONE

2. User login - DONE

3. Refresh JWT

4. User logout

5. Get current user's info

6. Create an idea

Delete an idea Get a page of ideas Update an idea

Current day and time: Fri Mar 8 2019 06:08

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr51m00s

Total time spent: 12hr41m34s

Delete an idea

Verify the idea id is valid & exists

Remove the idea from the db

Crap...

How do I make dynamic routes that have IDs in them in Flask???

Looking good.

I'm fucking hungry.

1. User signup - DONE

2. User login - DONE

3. Refresh JWT

4. User logout

5. Get current user's info

6. Create an idea

7. Delete an idea

Get a page of ideas Update an idea

Current day and time: Fri Mar 8 2019 06:25

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr15m00s

Total time spent: 12hr56m34s

We took a couple days off but

Get a page of ideas

first, log in, then pass the JWT to /getideas



{'jwt': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1NTIyMzkzMDMsIm5iZiI6MTU1MjIzOTMwMywianRpIjoiYzc3NGIyZWUtZTVmZi00ZTUzLWI2ZjktYjAyNDkxZWI1ZDI4IiwiZXhwIjoxNTUyMjQwMjAzLCJpZGVudGl0eSI6MSwiZnJlc2giOmZhbHNlLCJ0eXBlIjoiYWNjZXNzIn0.NJDvj57c_hxqoqt5N_0zPYFHqxJOOjhMdAaQ_jIIcR4', 'refresh_token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1NTIyMzkzMDMsIm5iZiI6MTU1MjIzOTMwMywianRpIjoiNzc5ZDYzZTctZjM0Ni00MjQ1LWFlMTEtODkwMDM0OGQ3OWYyIiwiZXhwIjoxNTUyMjM5OTAzLCJpZGVudGl0eSI6MSwidHlwZSI6InJlZnJlc2gifQ.xX9a60sgVZ01OHjFwSPFF8EV-E_oS1YBfLiglGECYL4'}

We are trying to get our Ideas back as json.



File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type Idea is not JSON serializable

So, how do we make an object in Python JSON serializable?

One workaround I got working is to call .serialize() once it is defined.



retval = jsonify([idea.serialize() for idea in idea_page.items])

I think that completes that!

1. User signup - DONE

2. User login - DONE

3. Refresh JWT

4. User logout

5. Get current user's info

6. Create an idea

7. Delete an idea

8. Get a page of ideas

Update an idea

Current day and time: Sun Mar 10 2019 13:54

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr20m00s

Total time spent: 13hr16m34s

Update an idea

DONE!!!

Current day and time: Sun Mar 10 2019 14:53

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr17m30s

Total time spent: 13hr34m04s

Ok, that seems to be everything.

We did omit the gravatar url thing.

We don't have enough time to submit now, BUT we are mostly done.

What you need to do:

Create a private Github or Bitbucket repository for your final code submission Add our Github or Bitbucket account to your repository Deploy your code to an online version Email us with the following information: Email subject:

Small Back End Project: xxxxx

Your name

Your repository link

An API server link where we can access and test

Gravatar URL

https://en.gravatar.com/site/implement/images/

Calculate email hash URL = https://www.gravatar.com/avatar/HASH

https://en.gravatar.com/site/implement/hash/

echo md5( strtolower( trim( "MyEmailAddress@example.com " ) ) );

Lets convert that to Python.



python3 import hashlib email = 'youremail@example.com' a = hashlib.md5(email.lower().encode()) a.hexdigest()

That gets what we want.



gravURL = 'https://www.gravatar.com/avatar/' + str(a.hexdigest())

We need this to happen at user registration.

This is in routes.py .

In order to do this, we need to update the User definition in the models.py file.

Or, could we return it calculated every time?

Maybe we can avoid updating the model.

Note to self: check the type before enforcing cast. Actually use your brain when writing code. I know we need to get shit done.

I anticipated this by having the field in getCurrentUserInfo .

Problem solved.

Submission time.

Create private github

sidenote: green tea time

done

Deploy our code to an online version

We are gonna try out the localtunnel method, it seems the least hassle. I don't feel like learning Heroku or whatever tonight.

Modern localtunnel:

Command is lt now.

Crap...just realized that I am going to have to install a lot of dependencies -_-;...



sudo apt install python3-flask -y python3-flask-sqlalchemy python3-flask-migrate python3-flask-jwt-extended sudo apt install python3-pip pip install setuptools pip install flask-jwt-extended

Ok, got it running on my server.

Looks like we have to do this instead:

I am very tired.

We will do this in the morning on a brand-new server.

Current day and time: Wed Mar 13 2019 4:39

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 1hr40m41s

Total time spent: 15hr14m04s

Back at it at night lol. ffff

Spinning up a server to host this thing on.

Goin apache mod_wsgi route.

First things first secure the server.

Done.

Next, install dependencies.



sudo apt-get install python3-pip \ python3-flask \ python3-flask-sqlalchemy \ python3-flask-migrate -y pip3 install setuptools pip3 install flask-jwt-extended

Done.

Now, all that is left is to move the app code into an appropriate directory and begin the wsgi setup.

Current day and time: Wed Mar 13 2019 22:31

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 0hr20m00s

Total time spent: 15hr34m04s

This is way too hard...

I've gone thru all the steps but keep encountering 500 errors when I try to serve it up.

I'm throwing in the towel.

Wait...

ITS SERVING!!!

WE DID IT!!!

FINALLY SUBMITTED EVERYTHING

FINAL TIME LOG!!!

Current day and time: Thu Mar 14 2019 2:59

Delivery date: Fri Mar 8 2019 11:59 PM

Current time spent: 1hr02m00s

Total time spent: 16hr36m04s

Over the course of like 2-3 weeks, managed to squeeze out 16 hours worth of extra "time" that was spent learning the "Flask" framework in a real crashcourse-like manner.

However.

I am now capable of developing and deploying web apps via Flask and mod_wsgi with JWT token login.

There's a lot to do in formatting this blog of materials, because it is really chaotic/scatterbrained, but I think a lot of people will find this useful and entertaining.

All URLs in this blog