Parsing AWS Public IP API with Python
In this video, we take a look at completing a network automation task from start to finish, including:
Workflow the problem
Develop the solution
Test the solution
Adding some optimisations
Repo: https://github.com/writememe/api_parser_demo
00:00 Introduction
01:12 Document the workflow
06:17 Create python virtual environment
07:06 Using a requirements.txt file
07:30 Installing modules from requirements.txt file
07:55 Setup a folder structure
08:48 Finding the AWS public IP ranges
09:10 Reading AWS public IP range documentation
10:00 Viewing AWS public IP JSON file
11:15 Documenting AWS public IP URL
11:33 Developing initial python solution
11:50 Solution disclaimer
12:25 Introduction Docstring Documentaton
13:44 Retrieving the URL
13:59 Python requests module
14:11 Importing requests module
14:24 Adding requests to requirements.txt file and install
14:48 Reading requests documentation and introducing concepts
17:03 Validating that requests is installed
17:19 Developing retrieve URL function
18:10 Docstrings for retrieve URL function
20:34 Using sample code from another project
21:22 Adjusting sample code from another project
23:30 Basic structure to retrieve_url function
24:44 Call basic retrieve_url function
25:15 Report URL success/failure
27:48 Assigning success/failure to a variable
28:27 If/Else logic for response success/failure
29:37 Wrapping main workflow into the main() function
30:45 Documenting resp_ok in docstring
31:44 Working with resp_ok in main function
32:57 Calling main function with resp_ok shell code
33:46 Debugging URL response output
36:14 Using requests JSON decoder
37:30 Document output variable
38:27 Dealing with multiple return values
41:42 Handling raw output when URL is unsuccessful
44:11 Find object type of output
46:32 Discussing return options
47:48 Documenting parse output function
49:28 Document the data structure which is being parsed
51:15 Refactoring the unsuccessful response
52:16 Building out parse output logic
53:25 Accessing basic values in parsed output
54:05 Accessing and printing the sync token and create date
56:05 Retrieve the “prefixes” and assign to a variable
56:32 Variable name discussion, continued
58:00 Printing out IPv4 prefixes
58:50 Iterating over IPv4 prefixes discussion
59:58 Create an empty dictionary to store results into
1:01:00 Adding a counter to make unique descriptions
1:02:03 Iterating over IPv4 prefixes list and access results
1:05:31 Printing out prefix results
1:06:08 Increment counter and print to screen
1:07:00 Generating an automated unique description
1:11:13 Append IPv4 prefix and description to the empty dictionary
1:12:35 Printout populated dictionary
1:13:40 Return dictionary outside function for usage for processing
1:16:38 Accessing IPv4 dict outside function
1:17:40 Output data structure to a file format
1:18:08 Param name discussion
1:19:20 Documenting output results function
1:20:43 Documenting return variables discussion
1:21:51 Setting up text_filename variable
1:23:33 Using with to context manage files
1:24:46 Inserting header to text file
1:25:35 Iterate over list and save values to text file
1:26:59 Format output line to write to text file
1:27:58 Write output line to text file
1:28:57 Return the filename and use outside function
1:31:14 Testing file outputs
1:31:32 Looking a text_filename output
1:32:04 Optimising the solution
1:32:22 Change text file output to tab deliniation
1:32:51 Using boolean flags for CSV/text output functionality
1:33:38 Documenting new params
1:34:34 Adding filename_prefix with default value
1:37:12 Return CSV filename docstring
1:37:37 Removing filename prefix and adding text if/else logic
1:39:00 Adding CSV if/else logic
1:41:50 Testing text/CSV with defaults
1:42:54 Testing with filename_prefix
1:44:20 Black reformatting
1:44:40 Pylama code audit/linting
1:45:36 Bandit code audit
1:45:56 Print statements cleanup
1:46:34 Testing final solution against workflow
1:47:30 Wrapping up
ipv4
