Automation Using Python Menu-Driven Program

 

Automation is the key to the future, as in industry we don't want any delay in providing service to the client and the implementation of automation technologies, techniques and processes improve the efficiency, reliability of many tasks that were previously performed by humans manually.

Today almost all the organization are using automation in their system and keeps on increasing on daily basis both in the software/hardware and machine layer.

In this article, we will see how automation reduces human effort and how the menu-driven program functions.

So let's explore the automated menu-driven program

I have created an automated menu-driven program using python it can perform most of the task of Hadoop, Linux, AWS CLI, docker, webserver and it will also configure yum repository automatically and most importantly it comes with a simple interface which makes it easy to use for those who don’t the commands of these technologies.

You just have to a Run command ->python3 interface.py in your system and relax it does everything you want in just a few steps.

Here is my GitHub source code of automated menu-driven program: https://github.com/agarwalyash02/CLI_automated_interface

This is the main menu of our program which is protected by a password, it gives a various option on which technology you want your work to be done.

Option 1 of the main menu — Performing basic Linux commands: There are two ways to login into the system locally and remotely so our program also asks for the same, if you have chosen to log in remotely it will ask the IP of the remote system.

after selecting the login type, then it will ask the task you want to perform in the Linux OS here is the list of options available.

There are various commands available which we use in our daily life all command usage is clearly given in the interface for the ease of understanding of the user, here we have also given the option of stopping the firewall but it is risky to stop the firewall, but sometimes when we some technology which blocks the connection between systems, in that case, we have to disable the firewall.

Option 2 of the main menu — Performing Hadoop configuration setup: Configuring Hadoop files is a very hectic task and if by mistakenly we have written a single word incorrectly then also our whole cluster of Hadoop won't work so it is very necessary to configure the files of Hadoop accurately but here our program will do everything for you without any error.

• Option 3 and 4 are there to start the service of the data node and name node
• option 5 can be used from all data node, namenode and client which show the status of our clusters like how many data nodes are connected and how much storage is available in the cluster and many more things
• option 6 is specifically only for the client, this option 6 is useful to seamlessly upload the files in the Hadoop cluster.
• There is more subcommand in the 1st and 2nd option which will ask for which we want to configure our files hdfs and core file.

All node has a different way of configuring hdfs file them so we have to created sub-options for different nodes, in namenode and data node it only asks folder name but in the client, it will ask for the replication factor and block size. And in the case of the core file configuration of data node and client is the same and the program only ask the port on which they want to run Hadoop services and IP of the master node but in the case of namenode, it only ask for the port number.

Option 3 of the main menu — Performing Linux partition: Adding a hard drive is not enough we have to follow some steps to store something in that storage

It is a one time 3 step process to make the drive accessible

• Step 1: We have to create a partition to make where you will get an option to give all the space to a single partition or you can multiple partitions, but there is a limit 4 for creating a primary partition due to limitations in the size of metadata table but there is also option to create a logical partition which will help to create more partitions
• Step 2: Every time we create a new partition we have to format that partition as it creates the Index table or Inode table which keep a record of all files with their stored location for easy access to that file.
• Step 3: User only have access to use only files and folder in an operating system, if we want to use the partition we have to mount it with a folder due to restriction of OS as the user can’t access any device in the system without the help of any program. When we mount all the storage of the device is allotted to that folder.

Option 4 of the main menu — Performing AWS CLI task: There are many ways to access AWS one of the common and easy ways is through AWS management console(WebUI) but some restrictions are working with the console as with console we are only able to launch one type of an instance at a time. And for automation, CLI is the best way to get things done.

• In this option 2 is used to check the status of the instances whether they are in running state or stopped and many more things
• option 3 is for the use of very great service of AWS CloudFront which uses a Content Delivery Network behind the scene which stores the caches in the nearby Edge location of the user for greater user experience
• option 4 is used to create the bucket using the S3 service of AWS in this we can store and it is mostly used to store all static data
• option 1 is used to launch a new ec2 instance we have to select which type of instance we want to launch and the type of instance and the rest of everything is done by the code.

We can see all the things are automatically done in just a few users input

Option 5 of the main menu — Performing docker tasks: Docker is a great tool it has many uses but the greatest thing about docker is that it can install any OS in just 1 sec and that OS consumes around 30- 40 MB of RAM, but if we do this same task without docker then for installing OS we need at least 1 GB of RAM and it takes around 30 - 40 min in installing a new OS but in the cooperate world wastage time is actually wastage of money so docker helps us in saving time as well as resources.

• This option 1 is to start, stop and check the status of the docker service as for running any service we have to start it using system service commands
• Option 2 is to check the running container and it also displays the id, name and the OS of all running containers where option 3 display all containers whether it is stopped or running.
• option 4 will create a new container in the docker, this program only ask for the name and OS you want to launch and all the things are done by the program itself whereas option 5 is to attach the running container with the current CLI of Base OS we only have to provide the name or container id of the container to attach.

Option 6 of the main menu — Performing Apache Webserver tasks: A web server is a computer that runs websites. It’s a computer program that distributes web pages as they are requisitioned.

• Option 1 is to write an HTML file that we want to host using the web server, the program only asks the name of the file which we want to create or edit it whereas option 2 is to start, stop, or check the status of the webserver.

Option 7 of the main menu — Performing yum related task: If we want to install something from the internet using yum we fir have to configure the yum repository after it repository gets configured we can install any software present in the yum repository and the best thing about yum is always downloading the latest and stable software for our OS and there is much software which depends on another software so yum install this software.

• Option 1 is configured yum repository with all software present in the DVD, it makes a new repo file where it set up docker repository and it also downloads epel-release software which makes more yum repository which helps install various other software.
• option 2 is to check whether the repository is successfully configured or not, this option also displays each repository with their software count whereas option 3 is only for the particular software whether it is present or not in our yum repositories.
• option 4 and 5 is to install and remove any software which is available in our configured yum repository.

Option 8 of the main menu — Create LVM partitions: LVM is a tool for logical volume management. With LVM, a hard drive or set of hard drives can be allocated to one or more physical volumes. LVM physical volumes can be placed on other block devices which might span two or more disks.

• This program automatically creates physical volume and volume group and create logical volume specified by the user and attach it with a folder provided by the user.

The goal of this project is to combine different technologies using python-script and make it more user-friendly.

In this automation, we have integrated python with different technologies like Hadoop, AWS, Docker, and we can integrate many more technologies also….

Thanks for reading this article! Leave a comment below if you have any questions.