Master Python Networking From A to Z - Part I: Fundamentals

';

Network programming has always been a demanding task. With full-featured and well-documented libraries all the way up the stack, Python makes network programming the enjoyable experience it should be.

Starting with a walk-through of today's major networking protocols, throughout this course, you'll learn how to employ Python for network programming, how to request and retrieve web resources, and how to extract data in major formats over the web. You'll utilize Python for emailing, using a variety of protocols, and you'll interact with remote systems and IP and DNS networking. The connection of network devices and configuration using Python 3 will also be covered.

As the course progresses, socket programming will be covered, followed by how to design servers and the pros and cons of multithreaded and event-driven architectures. You'll develop practical client-side applications, including web API clients, email clients, SSH, and FTP. These applications will also be implemented through existing web application frameworks.





This course is ideal for Python developers or system administrators with Python experience who are looking to take their first steps in network programming.

Python developers who are interested in going deeper into packages related to asynchronous programming would also benefit from this course. A basic knowledge of Python programming is recommended.

This is the first course of the series, which will help you master the basics of the network programming before moving on to more advanced courses…





In this course we will cover the following:





Network Programming with Python





This course will focus on writing programs for networks that use the Internet Protocol (IP) suite. Why have we chosen to do this? Well, out of the sets of protocols that are supported by the Python standard library, the Transmission Control Protocol (TCP)/IP protocol is by far the most widely employable. It contains the principal protocols that are used by the internet. By learning to program for TCP/IP, you'll be learning how to potentially communicate with just about every device that is connected to this great tangle of network cables and electromagnetic waves.

The following topics will be covered in this part:

· An introduction to TCP/IP networking

· Protocol concepts and the problems that protocols solve

· Addressing

· Creating RESTful web applications and working with flask and HTTP requests

In this part, we will be looking at some concepts and methods related to networks and network programming in Python, which we'll be using throughout this course.

This part has two sections. The first section, an introduction to TCP/IP networking, offers an introduction to essential networking concepts, with a strong focus on the TCP/IP stack. We'll be looking at what comprises a network, how the IP allows data transfer across and between networks, and how TCP/IP provides us with services that help us to develop network applications. This section is intended to provide a grounding in these essential areas and to act as a point of reference for them. If you're already comfortable with concepts such as IP addresses, routing, TCP and User Datagram Protocol (UDP), and protocol stack layers, then you may wish to skip to the second section, Python network programming through libraries.

In the second part, we'll look at the way in which network programming is approached with Python. This part provides a review of basic network elements and principles, as well as a discussion of how Python supports network programming with an overview of key libraries. Finally, we will introduce you to Wireshark, a protocol exploration and network programming diagnostic tool. We will also look at how we can interact with Wireshark from Python with the pyshark module.





Programming for the Web with HTTP

In this part, you will learn how to use Python as an HTTP client to make requests and retrieve web resources. You will be encouraged to try a series of example requests. We will compare urllib and the third-party requests library and show you the differences when working with forms and cookies. The third-party Requests package is a very popular alternative to urllib. It has an elegant interface and a powerful feature set, and it is a great tool for streamlining HTTP workflows. We also cover HTTP authentication mechanisms and how we can manage them with the Requests module.

The following topics will be covered in this part:

· Understanding the urllib package to query a REST API

· Understanding the Requests package to query a REST API

· Handling forms with urllib and requests with Python 3

· Handling cookies with urllib and requests with Python 3

· Handling HTTPS and HTTP Basic Authentication with requests

The urllib package is the recommended Python standard library package for HTTP tasks. The standard library also has a low-level module called HTTP. Although this offers access to almost all aspects of the protocol, it has not been designed for everyday use. The urllib package has a simpler interface, and it

deals with everything that we are going to cover in this part. The third-party Requests package is a very popular alternative to urllib. It has an elegant interface and a powerful feature set, and it is a great tool for streamlining HTTP workflows.





Application Programming Interface in Action





When we talk about APIs in relation to Python, we usually refer to the classes and the functions that a module presents to us for interact action. In this part, we'll be talking about something different, that is, web APIs.

A web API is a type of API that you interact with through the HTTP protocol. Nowadays, many web services provide a set of HTTP calls, which are designed to be used programmatically by clients; they are meant to be used by machines rather than by humans. Through these interfaces, it's possible to automate interaction with the services and to perform tasks such as extracting data, configuring the service in some way, and uploading your own content to the service.

The following topics will be covered in this part:

· Introduction to REST APIs

· Introduction to JSON and the json module

· Interacting with a JSON hybrid-REST API (Twitter)

· Introduction to XML

· Working with XML and a full REST API (Amazon S3 bucket) with the Boto module

You will learn how to use Python to extract data from the major data formats found on the web: HTML, XML, and JSON. An example of pulling useful information from a downloaded web page will be used to illustrate HTML, while interactions with REST APIs will be used to guide you through the essentials of working with XML and JSON.





Web Scraping with BeautifulSoup and Scrapy





When we want to extract the content of a web page by automating the extraction of information, we often find that the website does not offer any API to obtain the data you need and it is necessary to resort to scraping techniques to recover data automatically. Some of the most powerful tools can be found in Python 3, among which we shall highlight BeautifulSoup and Scrapy. Scrapy is a framework written in Python for the extraction of data in an automated way that can be used for a wide range of applications, such as the processing of data mining.

The following topics will be covered in this part:

· Introduction to web scraping

· Extracting information from web pages and parsing HTML with BeautifulSoup

· Introduction to Scrapy components and architecture

· Scrapy as a framework for performing web crawling processes and data analysis

· Working with Scrapy in the cloud