Created a Menu Using Python integrating AWS,Hadoop,Docker,Linux and Webserver
ARTH TEAM_TASK — 1
What is Automation?
Automation is a broad term that can cover many areas of technology where human input is minimized.Basic automation takes simple, rudimentary tasks and automates them. In our project,we automated technologies like AWS,Docker,Linux. Also we created a webserver and automated it. We could achieve all this using Python.
Why use Python for task automation?
The main reason is Python is a scripting language.We can read scripts and automate using it. Also Python is very easy to learn and offers great readability and approachable syntax. It resembles plain English, which makes it an excellent choice.
We created a python Menu driven program which runs linux commands.We used python speech recognition module so that we can speak what we need and the application recognizes it.Things we are able to achieve are:-
- Opening applications like chrome,notepad,media player etc
- Opening websites using chrome
- Some basic linux commands
Here we integrate some basic concepts of commands with python to automated the things.
if you haven’t watch the previous video watch it to clarify whether the commands are working or not. we will provide the github link to see the programming codes.
Let’s move to the AWS Part i.e. how we automate this also using python
After getting redirect from the main menu we are now on the AWS CLI part for the next automation
In this we used python speech recognition module so that we can speak what we need it recognizes and later on do the task for me
At first when we get on to this page it greets you and ask what to do ,if you want to know what tasks it performs just say anything including the help so that it can understand to help you.
then it will explain about its tasks ,mentioned below
here for demo i create a ebs volume , then for creating the ebs volume it requires the type ,size ,and region.
after taking the info it creates the volume for you in the region.
now if we want to see the volume we ask this code to show all the ebs volume created .It will redirect to the aws web and show all the ebs volumes
it can do everything which the code mentioned in the help section (mentioned earlier)
Here is the video in which i do the same steps mentioned above ,go through it AWS
Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications.
We created a menu driven program using python that can run commands of docker
This is the Menu
When we press option 1, we can install docker. The behind process is that it first creates a docker repo. Then install it using yum command.After that it starts and enables the docker service.
There are many images in Dockerhub. An image is simply a file that contains the OS, your executables, and any data files that might be related to your programs.In order to use images,we pull the required image from docker registry.
In our menu,we provided 4 options. First 3 options are linux based images.And we provided 4th option using which we can pull any other image.
In above two figures,we pulled alpine and ubuntu image.In below figure, we pulled a busybox image using 4th option
This option shows all available images in our system.
What is a Container?
A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.
Here,we created a container of an ubuntu image using option 6. The underlying command used here is:
docker run -it --name ubuntuos ubuntu
In the above example,we created a container.After we create,we get the terminal of it. Using the terminal,we can do required operations.
Another way is to create a container and run it in background.
docker run -dit --name ubuntuos ubuntu
Option — 4 and 5
Using the 4th option,we can list all running containers and using 5th option,we can list all running as well as stopped containers
Option — 8 and 9
We use above options to start and stop a container. To do that,we use docker start and docker stop commands
To stop a container, we use docker stop <os_name>
Option 10 and 11
Using this option,we can delete a container. While deleting a container we use docker rm <container_name>.
Using option 11,we can delete all containers. The command used here is
docker rm `docker ps -aq`
Using this option,we can delete an image.Here i deleted an ubuntu image.The command used here is docker rmi.
Hadoop is a framework that allows you to first store Big Data in a distributed environment, so that, you can process it parallely. In hadoop,we have two types of Nodes. They are Namenode and Datanode. Here,we constructed a python menu driven program to maintain hadoop system
Option 1 and 2
For setting up hadoop,we are using AWS. So we are checking if AWS is installed or not. If not installed,we can install it using option 2.
Press option 2 for installation of AWS CLI
Now, we have to check if hadoop is installed or not.
We use this option to install hadoop
Option 5 and 6
We use above options to create configure namenode and datanode
To check if namenode is stated,we can use jps command
Now,we use other option to start datanode.
And therefore,hadoop configuration is completed.
We configured a webserver using python. Our menu consists of :-
- Check if webserver software is installed or not
- If not, then install webserver software
- Check status,web service start or not
- If not, start the web service
- Create file which you want to show to the client
- Check list of how many files we have
- To view the website
In this option, we can check if the webserver software(here, we used htttpd) is installed or not.
If there is no such software,then we install it.We use yum package manager to install the software
Option 3 and 4
After installing the httpd software we need to start the service. First we will check if the service is started or not.For that we use below command:
systemctl status httpd
And to start the service, we use option 4. The underlying command is
systemctl start httpd
Option 5 and 6
We use above options to create and list files in /var/www/html directory. This is the main directory that contains all necessary files
By pressing this option,we host the web server.
This is our project github link: https://github.com/akshayvsuresh/arthtask
This project is done by Ayush Milan,Kritika Yadav,Vishnu Kumar,Akshay Suresh under the guidance of Vimal Daga Sir.