Python is a popular and powerful scripting language that can do everything; you can perform web scraping, networking tools, scientific tools, Raspberry Pi programming, Web development, video games, and much more. With Python programming, you can do even system programming regardless of the platform you are using.

When it comes to machine learning, there are Python libraries like TensorFlow, which powers many artificial intelligence projects.

You can perform natural language processing (NLP) using Python libraries like NLTK.

We will discuss the basic Python programming in this post, and on the future posts will build tools and see Python programming in action.

If you find the content of this post is a little tricky, tell me, so I start from zero level with Python programming in the upcoming posts since I assume this for beginners with very little understanding of what Python is.

Manipulating strings

Strings in Python are immutable, so they cannot be changed. Any change to a string’s contents requires making a new copy.

Dealing with string is very simple in Python.

String concatenation

str = "welcome " + "to Python" print (str)

We can also multiply a string like this:

String multiplication

str = "LikeGeeks" * 2 print (str)

Concatenate with non-string

You can concatenate with non-string by just converting the value to a string using the str() function like this:

str = "This is test number " + str(15) print (str)

Search for a substring

You can search for a substring using find method like this:

str = "welcome to likegeeks website" print(str.find("likegeeks"))

The find method prints the position of the first occurrence of the likegeeks string.

If nothing found, it will return -1 as a result.

The find function starts from the first character. However, you can start from the nth character like this:

str = "welcome to likegeeks website" print(str.find("likegeeks",12))

Since we start from the 12th character, the word likegeeks doesn’t exist from that position, so it will return -1.

Get substrings

So we got the index of the string we’re searching for, now we want to print that matched.

You can print string by index like this:

str = "first second third" print(str[:2]) print(str[2:]) print(str[3:5]) print(str[-1])

The first print line prints from the first character till the second character, while the second print line prints from the second character till the end. Notice the position of the colon. The count starts from zero.

If you use a negative number, the counting will start backward instead, like the fourth print line, which prints the last character in the string.

Replace strings

You can replace a string using the replace method like this:

str = "This website is about programming" str2 = str.replace("This", "That") print(str2)

If you have many occurrences and you want to replace the first occurrence only, you can specify that:

str = "This website is about programming I like this website" str2 = str.replace("website", "page",1) print(str2)

The first word only got replaced.

Strip strings

You can trim white spaces from a string using the strip method like this:

str = " This website is about programming " print(str.strip())

You can strip from the right only or left only using rstrip() or lstrip() methods respectively.

Change the character case

You can change the case of the characters if you want to compare them or something.

str="Welcome to likegeeks" print(str.upper()) print(str.lower())

Convert strings to numbers

We have the str() function, which casts the value to a string, but this is not the only cast function in Python programming.

There are int() , float() , long() and other cast functions that you can use.

The int() function cast the input to an integer, while float() function cast the input to float.

str="10" str2="20" print(str+str2) print(int(str)+int(str2))

The first print line just concatenates the two numbers without a summation, while the second print line adds the two values and print the total.

Count strings

You can use the min(), max() and len() functions to calculate the minimum character or maximum character value or the total length of characters.

str="welcome to likegeeks website" print(min(str)) print(max(str)) print(len(str))

Iterate over strings

You can iterate over the string and manipulate every character individually like this:

str="welcome to likegeeks website" for i in range(len(str)): print(str[i])

The len() function counts the length of objects.

Encode strings

If you are using Python 3, it stores all strings as Unicode strings by default, but if you are using Python 2, you may need to encode your strings like this:

str="welcome to likegeeks website" str.encode('utf-8')

Manipulating numbers

You can define numbers in Python like this:

a=15

Integers and floats the same way.

If you have float number you can round it, using int() function like this:

a=15.5 print(int(a))

Round numbers

You can use the round() function to round numbers like this:

a=15.5652645 print(round(a,2))

Just specify how many numbers need to round.

User-defined precision numbers

You may need to work with floating numbers that are of arbitrary precision. You can use the decimal module that handles numbers of user-defined precision.

First, import the module like this:

from decimal import * a=Decimal(5.5)

Generate random numbers

The random module in Python gives functions to generate random numbers.

import random print(random.random())

The generated number is between 0.0 and 1.0.

You can generate a random number from your choices like this:

import random numbers=[1,2,3,4,5,6,7] print(random.choices(numbers))

Python programming language provides a module called datetime that helps in handling dates, times.

import datetime cur_date = datetime.datetime.now() print(cur_date) print(cur_date.year) print(cur_date.day) print(cur_date.weekday()) print(cur_date.month) print(cur_date.time())

You can extract the value you need from the date, like the above examples.

You can get the differences between two times or two dates like this:

import datetime time1 = datetime.datetime.now() time2 = datetime.datetime.now() timediff = time2 - time1 print(timediff.microseconds())

The timediff variable is an object of type timedelta. However, you can create this kind of object yourself like this:

time1 = datetime.datetime.now() time2 = datetime.timedelta(days=3) time3=time1+time2 print(time3.date())

The strftime() method takes a format specification, and it formats the date or time based on that format.

And the following table specifies some of the format options that you can use:

%a Prints locale abbreviated name of the weekday. %A Prints the weekday. %w Prints the day of the week as a number. %d Prints the day of the month as a zero-padded number. %b Prints the month as a locale abbreviated name. %B Prints the month as locale full name. %m Prints the month as a zero-padded number. %y Prints the year as a zero-padded two-digit number. %Y Prints the year as a zero-padded four-digit number. %H Prints the hour (24-hour clock) as a zero-padded number. %I Prints the hour (12-hour clock) as a zero-padded number. %p Prints AM or PM. %M Prints the minute as a zero-padded number. %S Prints the second as a zero-padded number.

import datetime date1 = datetime.datetime.now() print(date1.strftime('%d. %B %Y %I:%M%p'))

You can use the strptime() function to create a date from a string like this:

date1=datetime.datetime.strptime(“2015-11-21”, “%Y-%m-%d”)

Also, you can create it like this:

date1= datetime.datetime(year=2015, month=11, day=21)

Dealing with the file system

Dealing with files is very easy in Python programming, believe it or not, this is the easiest language you can use to deal with files. You can say that Python is the easiest language in doing many things.

Copy files

The shutil module contains a function for copying files.

import shutil copied_path = shutil.copy('my_file.txt', 'copied_file.txt')

If my_file.txt is a symlink, the above code will create copied_file.txt as a separate file.

You can create a copy of a symlink instead like this:

copied_path = shutil.copy('my_file.txt', 'copied_file.txt', follow_symlinks=False)

Move files

You can move files from one location to another like this:

import shutil shutil.move('file1.txt', 'file3.txt')

You can rename a file using the rename function from the os module like this:

import os os.rename('file1.txt', 'file3.txt')

Read and write text files

You can use the open function to open files, and then use the read or write methods to read from them and write to them.

fd = open('file1.txt') content = fd.read() print(content)

First, we open the file for reading using the open function; then, we start reading the file content using the read function. Finally, we put the grabbed content into the variable content.

You can specify how many bytes you want to read for the read() function:

fd.read(20)

If the file is not too big, you can read the entire contents into a list, then iterate over that list to print the output.

content = fd.readlines() print(content[0])

You can write to a file by specifying the mode to open function like this. You have two modes of writing, the write mode and append mode.

This is the write mode where you will overwrite the old file content.

fd = open('file1.txt','w') content = fd.write('YOUR CONTENT GOES HERE')

And this is the append mode:

fd = open('file1.txt','a') content = fd.write('YOUR CONTENT GOES HERE')

Creating directories

You can create a new directory using mkdir function from the os module like this:

import os os.mkdir('./NewFolder)

This code will throw an error if the directory exists. Don’t worry; we will talk about exception handling in future posts so you can avoid such errors.

Get access & modification & creation time

You can use getmtime(), getatime() and getctime() to get modification time, access time and creation time respectively.

The returned time is formatted as a Unix timestamp; we can convert it to a human-readable format like this:

import os import datetime tim=os.path.getctime('./file1.txt') print(datetime.datetime.fromtimestamp(tim))

Iterating over files

You can use listdir() function from os module to get the files:

import os files= os.listdir('.') print(files)

Also, you can use the glob module to do the same thing:

import glob files=glob.glob('*') print(files)

You can write any extension for file globbing, like *.doc to get all word documents only.

Serializing Python objects

Serializing objects means converting a Python object to a byte stream for later reuse.

You can do that using the pickle module:

import pickle fd = open('myfile.pk ', 'wb') pickle.dump(mydata,fd)

You can deserialize this data using load() function like this:

import pickle fd = open('myfile.pk ', 'rb') mydata = pickle.load(fd)

Compressing files

The Python standard library enables you to work with different types of compressed files like tar, zip, gzip, bzip2.

To work with a zip file, you can use the zipfile module:

import zipfile my_zip = zipfile.ZipFile('zipped_file.zip', mode='r') print(file.namelist())

You can create a zip file from your files like this:

import zipfile file=zipfile.ZipFile('files.zip','w') file.write('file1.txt') file.close()

You can extract the zip file using extractall() method like this:

import zipfile file=zipfile.ZipFile('files.zip','r') file.extractall() file.close()

Also, you can append files to an existing zip file by using append mode like this:

import zipfile file=zipfile.ZipFile('files.zip','a') file.write('file2.txt') file.close()

You can compress gz or bz files using the gzip module or bz2 module.

import gzip import bz2 gz_file=gzip.GzipFile('files.gz','r') bz_file=bz2.BZ2File('fiels.bz2','r')

Then you can read and write in the same way.

You can deal with rar files using the unrar package. First, install the package:

pip install unrar

Then you can use it the same way.

import unrar.rarfile m=unrar.rarfile.RarFile('file.rar') m.namelist() m.extractall()

Parse CSV files

You can use pandas to parse CSV files. This package can parse CSV and Excel files, and extract data from it easily.

First, install the package:

pip install pandas

Then you can use it in your modules like this:

import pandas data=pandas.read_csv('file.csv)

Pandas treats the first column as a label for each row by default. You can pass the index_col parameter to specify the column index if it’s not the first column.

If there are no row labels in your document, you should use the parameter index_col=False.

To write to the CSV file, you can use the to_csv() method.

data.to_csv('file.csv)

Parse Excel files

You can use the read_excel() method from the pandas module to parse excel files.

data = pd.read_excel('file.xls', sheetname='Sheet1')

If you have multiple sheets, you can load it like this:

data = pd.ExcelFile('file.xls')

You can write to excel files like this:

data.to_excel('file.xls', sheet='Sheet1')

Networking and connectivity

The Python language has a socket class that provides a way of accessing the network at a low level; it has to support many networking protocols.

import socket host = '192.168.1.5' port = 5050 m_sock = socket.create_connection ((host, port))

This code we establish a connection to a host on IP 192.168.1.5 on port 5050.

After that, you can send and receive data.

m_sock.sendall(b'Hello World')

Notice that I used the b character before the string because data needs to be a byte string.

If you have a bigger message, you should iterate over your message like this:

msg = b'Longer Message Goes Here' mesglen = len(msg) total = 0 while total < msglen: sent = m_sock.send(msg[total:]) total = total + sent

For receiving data, you need to tell the methods of how many bytes to read in at a time.

data_in = m_sock.recv(2000)

This works because you know for sure that the message is less than 2000 bytes long.

If the message is big, you must loop over and over until you collect all of the separate chunks.

buffer = bytearray(b' ' * 2000) m_sock.recv_into(buffer)

Here we define an empty buffer; then, we start to write the message into the buffer.

Reading an E-Mail from POP mail server

We already discussed Linux mail server and everything about it, now, how we can access it using the Python programming?

The poplib module enables you to communicate with a POP server.

import getpass,poplib pop_serv = poplib.POP3('192.168.1.5') pop_serv.user("myuser") pop_serv.pass_(getpass.getpass())

The getpass module asks the end-user for passwords securely.

If you need a secure connection, use the POP3_SSL class instead.

To get the message list and the message count, you can do it like this:

msg_list = pop_serv.list() # to list the messages msg_count = pop_serv.msg_count() # to get message count

Don’t forget to close any open connections after you finish working with the POP server.

pop_serv.quit()

Reading an E-Mail from the IMAP mail server

You can work with an IMAP e-mail server using the imaplib module.

import imaplib, getpass my_imap = imaplib.IMAP4('imap.server.com') my_imap.login("myuser", getpass.getpass())

If you are using SSL on your IMAP server, you should use the IMAP4_SSL class instead.

To get a list of e-mails, you need to search:

data = my_imap.search(None, 'ALL')

Then you can iterate over the returned e-mail indices in the data variable and fetch the message.

msg = my_imap.fetch(email_id, '(RFC822)')

Finally, don’t forget to close the connection:

my_imap.close() my_imap.logout()

Sending an E-Mail

You can send E-mails using the SMTP protocol using the smtplib.

import smtplib, getpass my_smtp = smtplib.SMTP(smtp.server.com') my_smtp.login("myuser", getpass.getpass())

If you are using SSL on your SMTP server, you should use the SMTP_SSL class instead.

Once the connection opens, you can send the message like this:

Web crawling

To talk to a web server, you need to use urllib.request submodule.

import urllib.request my_web = urllib.request.urlopen('https://www.google.com') print(my_web.read())

Post to a web page

If you need to submit a web form, you know that you should send a POST request to the web page, and that’s what we will do.

import urllib.request my_data = b'Your Data Goes Here' my_req = urllib.request.Request('http://localhost', data=my_data,method='POST') my_frm = urllib.request.urlopen(my_req) print(my_frm.status)

Note that we can use mechanize or urllib2, there are many ways to achieve that.

Create a mini server

The socket class supports listening for incoming connections.

import socket host = '' port = 3535 my_server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) my_server.bind((host, port)) my_server.listen(1)

Now you can accept incoming connections like this:

addr = my_server.accept() print('Connected ... ', addr) data = conn.recv(1024)

Also, don’t forget to close the connection after you’ve finished.

conn.close()

Threading in Python programming

Threading is running multiple processes in parallel, it’s very useful, especially when you need to run a process in a different thread and keep your current thread unattached to avoid freezing.

The Python language includes a module called threading that contains a Thread class.

import threading def print_message(): print('The message got printed from a different thread') my_thread = threading.Thread(target=print_message) my_thread.start()

If the function takes a long time to finish, you can check to see whether it is still running or not by using the is_alive() method.

Sometimes your threads need to access global resources safely. You can do that using locks.

import threading num = 1 my_lock = threading.Lock() def my_func(): global num, my_lock my_lock.acquire() sum = num + 1 print(sum) my_lock.release() my_thread = threading.Thread(target=my_func) my_thread.start()

Using Raspberry Pi

With raspberry PI, you can create your technology. It’s a single-board computer with a low price.

You can use the Python module RPi.GPIO to work with the Raspberry Pi.

First, install the package in your Raspberry PI like this:

$ sudo apt-get install python-dev python-rpi.gpio

Now you can use it in your scripts. You can write output on the Raspberry Pi’s GPIO bus:

import RPi.GPIO as GPIO GPIO.setmode(GPIO.BOARD) GPIO.setup(1, GPIO.OUT, initial=GPIO.LOW) GPIO.output(1, GPIO.HIGH)

Reading from the Raspberry Pi’s GPIO

You can use the RPi.GPIO Python module to read data in from the GPIO like this:

import RPi.GPIO RPi.GPIO.setup(1, GPIO.IN) if RPi.GPIO.input(1): print('Input was HIGH') else: print('Input was LOW')

We’ve covered a tiny bit of Python; there is a lot to cover.

I promise you that I will do my best on the upcoming Python posts to cover the Python programming language basics, then we can start building awesome tools.

I hope you enjoy the post. Keep coming back.

Thank you.