Devops-blog

Computer Networking for devops

Vaseem Shaik — Sun, 29 Jun 2025 12:29:39 GMT

ab samajthey hey ae computer networking kiun important hey

as a devops engineer aapko kuch kuch task karni hothi hey - jaisey infra structure management

aapko vm setup karna hotha hey apna server setup karna padtha hey - us server mey internet routing karna padtha hey - us server ko proper connectivity milri hey kya , security milri hey kya , automation key thriugh create hora hey kya

devops engineer boltha hey ek server instance create kar saktha hun , ismey ipv4 use karoe , ipv6 jab devops confuse hojatha hey

phir ae aatha hey ki on primise ka setup jamana gaya

aws aap netwroking mey kaisa karogey hum olenge , train with shubha ney bola ec2 mey instance create karthey hey - client aakey bolega isko na aek vpc key andar dalo , hum phir sey confuse hojathey vpc ek private cloud hey internet kaisa milega vpc key andar aisey aapko bohot saare situations and scenarios aathey hey jismey networking bohot use aathi hey

How does the internet work ?

ae jo internet hey ae kaisa chaltha hey . - internet jabbi in logeney start kiya tha basically pura globe mey alag alag continent hey - in logon ko connect karnekey liye network hey one person to another person mai india mey hun sumiya poland hey unko kaisa connect karey us cheez key liye optic fibers use huey baically wires jaisa rehthey hey samundhar ey andar jakey jakey connect karwathey hey
pura world mey internet flow hotha hey data transfer hotha hey through optical fiber which is kept inside the river
ae optic fiber kiska hey - ae tier 1 companies ka big big companies ka hey - ab tier2 company bolthi hey bhaiyya aap ka jo optic fiber hey ne vo humko rent mey dedo kiunki hamara shyaam vo usney bandhi sey bath karni hey ae tier1 ney $$ mey tier 2 ko rent ko dey dethi hey ae tier2 ney tier3 like vodaphone in companies ko dethi hey
why do you prefer optical cable over satilite
- because distance
ae kaisa access hotha hey - samajlo ek badasa duniya hey - tier1 sabka jaal bana key raka hey - tie2 rent lekey rakha hey tier3 companies like codaphone , airtel insey lekey humsey paisa lekey internet server kartha hey data provide karra hey
tier 3 companies ae kaisa kaam karthey hey

aisa samjlo ki aas pass hamara dost rehtha hey u broadband , jio , airtel broadband ae aapko internet deneykey liye tayar hey - ae ek isp hey
ae basically ek level pey kaam karra hey ae city key level key level pey aapko network provide karra hey usko bolthey hey MAN → metropolitan area network
agar aehi cheez lets say u broad band airtel sey lera hoga airtel tier2 company hey ae , aisa b ho saktha hey u broadband airtel key pass jatha hey boltah hey ki hu,mko rent pey dedo humsey thoda paisey lelo aisa airtel kithney ko b dey sakthey ae WAN wide area network mey kaam kartha hey bada hey na badi comapny bohot saarey logon ko dey sakthey hey
ab aapka isp aapkey gharthak internet paunchatha hey ae aatha hey LAN - local area network

OSI Model : full form → open system inter-connection ab samajthey ae osi kya hey

aisa samajlo ae aek logical kind of standard set kar diya hey
ae kaisa chaltha hey - chalo nibba and nibbi hey apney
nibba and nibbi kaisa bath karrey hey ae ho kya raha hey - kounsa application mey bath karrey hey for ex: whatsup
aap kisi nibba nibbi ko message bhejthey ho - aapko ek msg dikhatha hey uppar end to end encryption
beech mey na presentation layer rehthi hey inki bathon ko format karthi hey encrytp karthi hey thaki inki bathon ko na koi b na sun sakey kiunki ae log privacy sey bath kar sakey
jab nibba and nibbi online dikhthi hey thab message kar sakthey hey in dono ko beech mey session maintain hothi hey - jab dono online mey ho
jab inkey dono neech mey dil ki bath aajthi hey transport hojath hey nibba nibbi ki bathey
pehley three layers are software lawyer - ek baar transport honey laga phir jo messages janey lagey ne vo internet through jayega iskey badh network layer hey - network toh milgaya ab nibbi ka address kya device address - yaa data-link layer - ek baar dil judgaya phir nibbi key pass wifi connectivity - router physical layer honi chahiye nibbi key pass agar wifi nahi hey aa kithey b messages bhejo nahi reach hoyenge usko
ae transport sey na hardware lawyers hey
nibbi key pass physical layer honi chahiye phir device address i mean data -link - phir network hona chahiye

Soft of OSI

ae 7 layers kaisey kaam karthey hey - ae jo application layer hey ae ui hotha hey vo browser hotha hey jispey aap basically internet ko access karthey hey

phir aapka presentation layer hotha hey aapka jo data alag format hey hotha hey , network ko thoda jumbled way mey samaj mey aatha hey isko format kartha hey presentation layer

phir session layer aap key pass ek client hey - aap aur aap key client key beech mey session maintain rakhna hey jo break na ho isliye session layer ek security maintain kartha hey sirf dono ko beech mey ae sab rehtah hey software part phir aatha hey

Hard of OSI :

transport layer - basically data ko transport kartha hey using some types of protocals , matlab set of rules is device sey us device ko bhejna hey kuch set of rules hey for ex: tcp /ip → transmission conrol protocal or internet protocal ae kuch rules hey jiskey vajhasey vo data transfer ho patha hey isliye ae center mey rehtah hey
ek baar aapka data transfer honey laga - aapko vo data transfer honey key liye aapko ek network lagtha hey toh aapko chahiye network layer is network layer mey baically saarey protocals rehthey hey [ip] - internet protocal for ex: file transfer karni hey iskey liye particular tyr ka protocal , dns ko , email ko is cheezon ko particaular type of protocals rehthey hey phir aatha hey
data-link - ae aisa samajlo network decide hogaya but vo jo nibba and nibbi hey unkey pass jo device hey unkey device ka mac address , ip address kya hey vo data link layer pey ip , mac address ae sab patha chaljatha hey jiskey liye alag protocals rehthey hey like [ARP , RARP]

mac ko find kartha hey arp and ip address ko find kartha hey using RARP.
last ab cables aathey hey physical layers mey aathey hey , cables , NIC Network integrated chip , wifi , lan cables.

AB SAMAJTHEY HEY TCP/IP MODEL

OSI model aek standard iso ney standard bana diya tha but reality ki bath aathi hey ek model use aathi hey TCP/IP model hi use hothi hey
osi ek logical model -jo real life mey use karthey hey in the way of tcp/ip model both are same .
[ TCP / IP ] -

1. applicaton layer → ismey aapka ui , broser , snapchat , whastup rahega but usmey protocals use hothey hey vo b mention kara na hey web pages browse hothey hey html mey jab b browser mey http , agar mail karrey ho - smtp -simple mail transfer protocal ae sab cheeze batani honi chahiye
  1. Transport layer → jaha pey actual mey aapka data tarnsfer hoga - ek nibba hey usney nibbi ko hi bheja uskey jagah jakey hi chudel karkey gaya isney bheja hi dear - nibbi key pass pahuncha hi chudel ae aisa na ho is layer mey error checking , checksome , packets kithey transfer horehey , ae packets securely jarehey yah nahi , is cheez key liye transport layer mey protocals hey jaisey TCP - transmission control protocal aek secure connection banadetha hey for ex:
    
    ek nibba ney ek nibbi ko msg bheja - ae msg gaya , nahi gaya is cheez ka ek acknowledgement aatha hey agar vo acknowledgement nibba key pass aayage thab hi aagey badega ae tcp hey na ae aek three way handshake and secure protocal hotha hey ae transport layer mey hotha hey
    
    omegel mey UDP → user datagram protocal mey aapka nibba aapkey nibbi ko msg bhejtha rahega agar pahuncha toh theek hey nahi toh koi bath nahi yaha secure nahi hey send hojatha hey response miljatha hey basically acknowledgement wala thing nahi rahtha
    
    udp connection less hotha hey

Network layer / internet layer : pey aapkey ip address pey bath hori hothi hey - aap ka source ip kya hey , destination ip kya hey in sab cheezon pey bath hothi hey
- jab mai www.google.com - > first application layer hey ae phir iskey badh transport layer baically data ghoomkey jayega phir iskey badh network layer aathi hey ismey source ip and destination ip kaisa patha karthi hey isko trace arna hey traceroute google.com → ae hey mera ip iskey badh ip hey internet service provider
- ip locater → sourceip find → yaha sey request kaha pey gayi isp pey → phir vaha sey mumbai aaya ae tcp hey ab ae california ko gaya vaha google ka server pey gaya → kuch packet drop nahi hua
- ab samajthey hey Network access layer hey vo kya kartha hey hamae transport layer samaj mey aagaya
- address resource protocal [arp] aapki source ka ip , mac ip and , reciever ip bhejdetha hey lekin mac ip patha nahi hotha hey is cheez ko na braodcast kar detha hey ip address key sath network mey jo registered address dikha detha hey

Mac address: ek physical device address hotah hey thaki recognize kar sakey , agar mac address patha karna hey isliye ek protocal rehtha hey arp → address resource protocal -→ mac address remains same - bacially aap ki ip address key sath ek mac address jud jatha hey

abhi mummy and papa tv dekhre merko unka mac ip patah karna hey mai apney roter mey login karunga , 198.168.0.1 -→ login karo ae karnekey badh mere ghar ka computer networking chalra hey dhcp clients mey bata tha hey ki koun koun is router mey juda hua hey - ip address aek logical address , mac address ek physical address hey

kabhi kabhi mac address appko patha nahi rehta isliye arp - mujhey mera mac ka patha nahi toh mai router ko bolunga mera mac ka mac address dedo mai isko bolunga mujhko apna id address dedo us ip ko mai broadcast karunga ussey jo b connected hey ame network mey hey mujhko uska mac address miljayega vo mac address mai aapko dedunga

by roadcasting it will braidcast and by unicasting it will send mac address to you personally .

CAN - campus area network

dhcp: DYNAMIC HOST CONFIGURATION PROTOCAL - through this we can restrict the access , we can give the access .

RARP: same tharikey sey RARP same process hotha hey ip address patha nahi rehtha vo miljatha hey .

IMPORTANT PROTOCALS :

1. dns - domain name system : jab b aap likhthey ho www.trainwithshubham.com ae ek domain name iskey background mey ek server hey mai server mei jaunga nahi mai sirf tws.com likhunga ae tws domain map kiya gaya hey aek ip sey aek address sey map hp chuka hey
  - go to go daddy - ae ek platform where we can register our domain name - here now i can purchase the domain mainey aws pey yah kahi pey server liya aur us server oey us website dali , ab aisa kuch ho mai tws.com lu background mey server khulgaye ae kartha hey godaddy - go to manage dns and -dns records - is address book mey ek domain lekey address assign kardetha hun
  1. FTP: [file transfer protocal] -> aek aisa connection bitaletha hey client ki thaki ek sercurely files ko transfer kar pau .
    - index of games - toh ek ftp ka server hey
    - mai ek ftp client like filezilla leletha hun aur jo b merko chahiye from sever download kar letha hun
  2. telnet : (terminal network ) - ae first protocal tha jissey ki aap kisi b remote ko acess karthey they
    - mujhey yaha baitkey kisi remote server ko connect karna hey yah toh mai ssh , or telnet sey b kar saktha hun telnet sey ui b dekhsaktha hun
  3. SMTP : [ simple mail transfe protocal ] - aap ubuntu mey ubuntu smtp setup sendemail
  4. SNMP [ simple network management protocal] - kisi network ko manage karna hey snp use karthey hey bohot rarely use hotha hey
  5. http [ hyper text transfer protocal ] - api calls , jithey b web pages load , request api calls hothey hey http protocal through .
    - http pa port number 80
    - https ka 443
    - → ae http methods bohot important hey ae client and server https through aek server sey bathey karna chahtha hey
    - client sends request and server sends responce what he wil talk , either he wants to read , write , update , and delete the data
    - agar aapko read karna hey get method , agar write karna hey post method , agar update karna hey - put method and delete karna hey - delete method .

Data-link + physical layer combines = Network access layer

1. Network access layer :

AWS Practical :

Is duniya mey kithna sara internet hoga , kithna sara ip address hoa , kithney saare ip address hogi
ae mera ghar mai mummy , papa , didi mere pass ek phone hey ek lap hey ek har , mummy , papa , and didi jey pass mobile hey aek har ek devices ko ek ip assign hotha hey aisa ek gharmey 7 devices ip address aisey ek mohelley mey kithne ghar honge
aapmey shehar mey kitne honge aisey kithne shehar kithne honge aise aise shehar kithey countries mey honge ithne sarey ip address manage nahi kar sakthey toh

IPV4 - internet protocal version 4 limited ip address detha hey -→ no of ip address in ipv4 424 croces agar peoples badh gaye toh thab aaya ipv6 ae bada internet hey ismey chota chota internet banayenge like subnets is subnets mey aap ip address banavo subnet mey ip address unique honge

ae jo vpc hey ae aapka personal network hey - is ko design karn ahey is vpc ko aap ek region mey banathey ho , ismey vpc daloge ismey subnet dalogey basically chota chota internet , agar aapka koi device hey public ko face karani hey vo hoga aapka public subnet agar aapka kuch data base hey privately rahna hey toh aapka private subnet

ab ae toh ban gaya is vpc mey internet kaisa ghuseka iskey liye ek internet gateway banan hey thaki vpc mey internet ki acacces hojaye , bahar ki duniya sey bathey hopaye isliye gateway chahiye
is public subnet mey ek ec2 instance hey , private subnet mey rds hey
ae public subnet aur private subnet ko kaisa patha chalega ki internet ko kaha route karana hey , yah phir kis ip address ko kya routing honi chahiye is cheez key ek route table b chahiye hoga jaha pey aap apney public aur private subnet ka info dedoge
ki bhaiyya is private subnet mey jithney b hey sab ko internet dedo -→ agar aapko internet access karana hey OSI model ko follow karna padtha hey → physical karwana hey
mai devops engineer mey task mila hey ujhey secure tarikey sey secure instances chalana hey mujhey sabsey vpc mey chalo → region select karlo - us region mey vpc create karunga → vpc and more → name tws-networking
ab preview dekho aapka jo vpc name - uskey andar kithne subnets banana hey humko delect karna hey
ab question aatha hey ab CIDR BLOCK kya hotha hey - us subnets mey jithney b devices rahenge un sabko ip address kya kya assign honge → classless interdomain routing —> basically aapkey subnets mey kithney ip address assign kiya ja saktha hey ek notaton rehta hey —> google karo cidr xyz
will i really get 65536 ips no aws khu 5 ips letha hey khud support key liye detha hey
pehley vpc phir , subnets ismey a.z phir subnets ko internet chahiye na jithney b ips hey isko route chahiye na isliye public ko ek route table private 2 ko 2 private route tables
internate gateway rehtha hey public subnet key liye
s3 gateway hoga private subnet key liye → create vpc

now create the instane in under your vpc and give info based on your requirement.

Hin-English Explanation

Kyun Computer Networking Important Hai for DevOps?

Ek DevOps engineer ke liye networking bohot zaroori hai kyunki aapko infrastructure manage karna padta hai. Jaise:

VM ya Server Setup: Aapko virtual machines (VM) ya servers setup karne padte hain. Isme internet routing, connectivity, aur security ensure karna hota hai.
Automation: Servers ya instances ko automate karna padta hai, jaise provisioning ke through.
IPv4 vs IPv6 Confusion: Jab aap ek instance create karte ho, toh IPv4 ya IPv6 ka use hota hai. IPv6 thoda complex lagta hai kyunki isme zyada IPs hote hain.
On-Premise vs Cloud: On-premise setup mein networking ka kaam alag hota hai, aur cloud (jaise AWS) mein VPC (Virtual Private Cloud) ke saath kaam karna padta hai. Client bolta hai, “Isko VPC ke andar daal do,” aur yeh confuse karta hai kyunki VPC private hota hai, toh internet kaise milega?

AWS mein Networking Kaise Kaam Karta Hai? AWS mein, jab aap ek EC2 instance create karte ho, client bolta hai, “Isko VPC ke andar daal do.” VPC ek private cloud jaisa hai, jisme aap subnets banate ho (public aur private). Public subnet internet se connect hota hai, private subnet secure rehta hai (jaise database ke liye). Internet access ke liye Internet Gateway banate hain, aur routing ke liye Route Tables use hote hain jo decide karte hain kaunsa traffic kahan jayega.

Internet Kaise Kaam Karta Hai?

Global Connectivity: Internet pure globe ko connect karta hai. Alag-alag continents ke beech optical fiber cables hote hain, jo samundar ke neeche bichhe hote hain. Yeh cables data transfer karte hain.
Tier System:
- Tier 1 Companies: Yeh badi companies hoti hain jo optical fibers ka network banati hain aur own karti hain.
- Tier 2 Companies: Yeh Tier 1 se fibers rent pe leti hain aur Tier 3 ko deti hain.
- Tier 3 Companies (jaise Airtel, Vodafone): Yeh ISPs (Internet Service Providers) hote hain jo humein internet provide karte hain. Yeh city level pe kaam karte hain aur MAN (Metropolitan Area Network) banate hain.
LAN, MAN, WAN:
- LAN (Local Area Network): Ghar ya office ke andar ka network.
- MAN: City-level network jo ISP provide karta hai.
- WAN (Wide Area Network): Bada network jo multiple cities ya countries connect karta hai.
Optical Fiber vs Satellite: Optical fiber zyada pasand kiya jata hai kyunki satellite se distance aur latency zyada hoti hai. Optical fiber fast aur reliable hota hai.

OSI Model Kya Hai? OSI (Open System Interconnection) ek logical standard hai jo networking ko layers mein samjhta hai. Yeh 7 layers hote hain:

Application Layer: Yeh UI hota hai (jaise browser, WhatsApp). Yahan HTTP, SMTP jaise protocols kaam karte hain.
Presentation Layer: Data ko format aur encrypt karta hai (jaise end-to-end encryption WhatsApp mein).
Session Layer: Client aur server ke beech session maintain karta hai (jaise jab dono online hote hain).
Transport Layer: Data transfer karta hai using protocols jaise TCP (secure, acknowledgement deta hai) ya UDP (fast, no acknowledgement).
Network Layer: IP addresses aur routing handle karta hai (jaise source aur destination IP).
Data-Link Layer: Device ke MAC address aur IP address ka pata lagata hai (ARP, RARP protocols use hote hain).
Physical Layer: Cables, NIC (Network Interface Card), Wi-Fi jaise hardware hote hain.

TCP/IP Model Kya Hai? TCP/IP model OSI ka practical version hai jo real-world mein use hota hai. Iske 4 layers hote hain:

Application Layer: Browser, WhatsApp, email (HTTP, SMTP protocols).
Transport Layer: Data transfer ke liye TCP (secure, three-way handshake) ya UDP (fast, no acknowledgement).
Internet Layer: IP addresses aur routing (source aur destination IPs).
Network Access Layer: Data-Link aur Physical layer combine hote hain. Yeh MAC address, cables, aur hardware handle karta hai.

Important Protocols:

DNS: Domain names (jaise trainwithshubham.com) ko IP address se map karta hai.
FTP: Files securely transfer karta hai.
Telnet/SSH: Remote server access ke liye.
SMTP: Email transfer ke liye.
SNMP: Network management ke liye (rarely used).
HTTP/HTTPS: Web browsing ke liye (port 80 aur 443).
ARP/RARP: MAC address aur IP address find karne ke liye.

AWS Practical:

VPC Setup: AWS mein VPC create karte ho ek region mein. Isme subnets banate ho (public aur private).
CIDR Block: Yeh decide karta hai kitne IP addresses ek subnet mein honge (jaise /16 mein 65,536 IPs, lekin AWS 5 IPs reserve karta hai).
Internet Gateway: Public subnet ko internet se connect karta hai.
Route Tables: Decide karte hain kaunsa traffic kahan jayega.
S3 Gateway: Private subnet ke liye S3 access ke liye.

Example:

VPC create karo (name: tws-networking).
Subnets banao (public aur private).
Internet Gateway attach karo public subnet ke liye.
Route Tables banao aur IPs assign karo.
EC2 instance public subnet mein aur RDS private subnet mein rakh do.

Full English Explanation

Why is Computer Networking Important for DevOps? For a DevOps engineer, networking is critical because it underpins infrastructure management. Key tasks include:

VM and Server Setup: Setting up virtual machines (VMs) or servers requires configuring internet routing, ensuring connectivity, and securing the infrastructure.
Automation: Automating server provisioning and instance creation is a core responsibility.
IPv4 vs IPv6: DevOps engineers often work with IPv4 for simplicity, but IPv6 can be confusing due to its vast address space.
On-Premise vs Cloud: On-premise setups involve physical networking, while cloud environments (like AWS) use Virtual Private Clouds (VPCs). Clients may request instances to be placed in a VPC, which can be confusing as VPCs are private by default, raising questions about internet access.

Networking in AWS In AWS, you create EC2 instances and place them in a VPC, which acts like a private cloud. A VPC contains subnets:

Public Subnet: Connects to the internet (e.g., for web servers).
Private Subnet: Isolated for secure resources (e.g., databases). To enable internet access, you attach an Internet Gateway to the VPC. Route Tables define how traffic is routed between subnets and the internet.

How Does the Internet Work?

Global Connectivity: The internet connects the globe via optical fiber cables laid under oceans, enabling data transfer across continents.
Tier System:
- Tier 1 Companies: Own and manage the global optical fiber network.
- Tier 2 Companies: Rent bandwidth from Tier 1 and provide it to Tier 3.
- Tier 3 Companies (e.g., Airtel, Vodafone): Act as Internet Service Providers (ISPs), delivering internet to end-users at the city level, forming a Metropolitan Area Network (MAN).
LAN, MAN, WAN:
- LAN (Local Area Network): Connects devices within a home or office.
- MAN: Covers a city, managed by ISPs.
- WAN (Wide Area Network): Connects multiple cities or countries.
Optical Fiber vs Satellite: Optical fibers are preferred over satellites due to lower latency and higher reliability. Satellites have higher latency because of the distance signals must travel.

What is the OSI Model? The OSI (Open System Interconnection) model is a logical framework that standardizes networking functions into 7 layers:

Application Layer: Provides user interfaces (e.g., browsers, WhatsApp). Uses protocols like HTTP, SMTP.
Presentation Layer: Formats and encrypts data (e.g., WhatsApp’s end-to-end encryption).
Session Layer: Maintains sessions between client and server (e.g., ensuring both are online).
Transport Layer: Handles data transfer using protocols like TCP (secure, with acknowledgements) or UDP (fast, no acknowledgements).
Network Layer: Manages IP addresses and routing (source and destination IPs).
Data-Link Layer: Handles device addressing (MAC and IP addresses) using protocols like ARP (Address Resolution Protocol) and RARP (Reverse ARP).
Physical Layer: Involves hardware like cables, Network Interface Cards (NICs), and Wi-Fi.

What is the TCP/IP Model? The TCP/IP model is a practical implementation of the OSI model, used in real-world networking. It has 4 layers:

Application Layer: Includes user interfaces and protocols like HTTP, SMTP, and FTP.
Transport Layer: Manages data transfer with TCP (reliable, with three-way handshake) or UDP (fast, connectionless).
Internet Layer: Handles IP addressing and routing.
Network Access Layer: Combines Data-Link and Physical layers, managing MAC addresses, cables, and hardware.

Key Protocols:

DNS (Domain Name System): Maps domain names (e.g., trainwithshubham.com) to IP addresses.
FTP (File Transfer Protocol): Enables secure file transfers.
Telnet/SSH: Allows remote server access (SSH is more secure).
SMTP (Simple Mail Transfer Protocol): Used for email transfer.
SNMP (Simple Network Management Protocol): Manages networks (rarely used).
HTTP/HTTPS: Powers web browsing (ports 80 and 443).
ARP/RARP: Resolves MAC and IP addresses.

AWS Practical Networking:

VPC Creation: Create a Virtual Private Cloud in a specific AWS region. A VPC contains subnets (public and private).
CIDR Block: Defines the range of IP addresses for a subnet (e.g., /16 provides 65,536 IPs, but AWS reserves 5 IPs for internal use).
Internet Gateway: Connects public subnets to the internet.
Route Tables: Define traffic routing for public and private subnets.
S3 Gateway: Provides private subnets access to S3 without internet exposure.

Example AWS Workflow:

Create a VPC (e.g., name: tws-networking).
Define subnets (public for EC2 instances, private for RDS).
Attach an Internet Gateway to the VPC for public subnet connectivity.
Configure Route Tables to route traffic appropriately.
Launch an EC2 instance in the public subnet and an RDS instance in the private subnet.

This explanation covers the importance of networking, how the internet functions, OSI and TCP/IP models, key protocols, and practical AWS networking concepts, presented in both Hin-English and full English for clarity. Let me know if you need further details or specific clarifications!

Kubernates Basic to Advance

Vaseem Shaik — Wed, 18 Jun 2025 09:33:01 GMT

Kubernates

Google ney 2014 mey google key websites scale nai ho parey - system robust nahi tha , vo ek time key badh crash hojarithi
2014 mey un honey ek system banaya borg jithney b applications hey vo automatic heal , scaleup and inhoney kubernates ka initial name borg tha phir ae opensource kar diya
Kubernates ia as orchestration tool hey , aapkey jo docker containers rehthey hey ae saare docker containers ek dusrey sey bathey karthey hey , kab kisko heal hona hey , scale hona hey ae sab kubernates karatha hey
CNCF is an organization , its a community

Kubernates kiun use karthey hey - jo bohot saarey applications monolithic banthey they ab microservice use karrey

Monolithic and microservice.

Monolithic - > sambo ek dmart hey - badi si dukhan hey sab miljayega sab kuch ek hi jagah miljayega , monolith ek bada application ek hi application mey saari ek hi repo mey daley hey , is mey problem 1. manage karna difficult hey - stoc mey ek service ghadbad hogai - pura ka pura application shutdown hojatha hey - socho ek thodi ki cheez update karna hey toh saara application update karna padtha hey , manage karna bohot difficult hey

solution is Microservice- chotey chotey store- multiple stores like micro services - badey application ko chotey chotey services ko divide kar diya -

microservices ko manage karna easy hey - chotey services docker key through chala sakthey hey - docker key through chali hui service ko manage and orchestrate karna hey aap kubernates use kar sakthey hey .
aap ko pata chalna hey ki aap kiun sikhre - kiunki microservice architecture bohot zyada use karrey isliye

Kubernates Architecture:

Kubernates ka logo kiun wheel jaisa - ship ka captain steering wheel hey na ae kubernates chalatha hey
aap kisi company mey jatha hey - startup matlab docker container - na scale kar patha hey - kabhi chaltha hey kabhi down hojatha hey
samjo mnc - mnc ka kuch architecture hotha hey - kubernates ka jo architecture hotha hey usmey aek aisa server rehtha hey - jo bakhi servers ko manage kartha hey
Cluster k8s mey hum cluster pey kaam karthey hey - multiple node ho sakthey hey - ek node hey master - dusrey nodes hey - worker nodes
mnc sey pov mey dekho - uska jo headquarter hey kahi dhoor rehthea hey - headquarter newyork - yaha pey actual kaam nahi hotha hey , vaha pey manager ceo log baidthey hey
india mey pune 1 node , banglore node , noida node ae node pey actuall log jakey work karthey hey
kubernates mey b aisa ek architecture rehtha hey - ek master node rehtha hey jaha pey ek aisey services hey jo worker node pey kaam karvaney key liye baiti hui hey - worker node mey actually docker key container chalre honge ae chalva rahehey vo master node pey baitey huey rehthey hey

master node mey kuch services rehthey hey

- apiserver: jo purey workernode mey kya kya cheeze chalri hey , master node mey kya kya cheeze chalre hey vo ek info rakhtha hey like Team lead - aapka team lead kahi dhoor bairta rehtha hey mumbai yah phir delhi vaha hey aapkey manager yah aapko kaam batata rehtah hey - api server ek aisa unit service hey jo communicate gateway rehtha hey har ek component key sath , aapkey team lead bola ki hamara requirement hey ek devops person ki jo ,placement karwatah hey vo HR - SCHEDULAR
  
  * schedular: ek aisi service hey schedular aapkey containers ko woker node pey scheduler karwathi hey , ab hr ney toh opening nikaldhi aapko job lagwadi , aapka jo data rahega ki aapney kab vo pod create kiya , kab opening hui kab job mili ae saara data base rehtha hey us db ko kubernates mey bola jatha hey - etcd
  
  * etcd: ae ek db hey - jo b cluster mey chalra sab data information etcd mey rehtha hey - data store rehtha hey , ab ek vo rehtha hey na pura ka pura bada sa head manager - dekhtha hey ki merey sare location mey kaam hora hey yah nahi hamra project manager
  
  * Controll manager : dekhtha hey sab saarey achesey chalre yah nahi - vaisey controll manager rehtah ehy - ae dekhtha hey ki , pods, services, nodes, cluster sahis ey chalra yah nahi dekhtha rehtha hey
Worker node jaha pey actual mey kaam chalra hoga udhar mey b kuch services rehthey hey like udhar b manager rehtha hey
- kubelet: jo ensure kartah hey aap aarey yah nahi , aap task karreho yah nahi , sahi sey login karra, sahi sey task submit karra yah nahi , kubelet apiserver ko bata tha hey - api sever controll manager ko bata hey aur db mey save kartha hey , pods isolate rehtha hey usko access nahi karsakthey hey , ek proxy rehtha hey idhar ka udhar chulbuli kartha hey
- service or kube proxy: as a user api server ko bolkey , kubelet ko bolkey andar key pods ko access kar sakthey hey
- CNI network : master to worker node communication chalatha hey
- kubetcl - director ceo - aisa karo vaisa karo karkey bata tha hey
  
  aapkey k8s ko instructions dena hey , dikhav kithey node chlarehey kithne pods chalre - pure key pure cluster ko aadesh dey sakthey hey
- pod: pod is a smallet unit in kubernates in which dokcer containers will be running , pod to pod communicate we make through service

Setup On Local:

Kubernates Installation in various types
- kubeadm
- Miniqube
- KIND Cluster
- Eks/Aks/Gke
Rancher - enterprice , Civo - ae sab like aws - enterprice ka hey
t1.medium lelo - connect karo - kind cluster - google karlo - or Kubestarter repo mey Kind-cluster mey jakey copy karkey install kardo
cluster create karey kind mey yaml likhna padtha hey - yaml - after configuring yaml
- kind create cluster —name=tws-cluster —config=config.yml
- kubectl cluster-info --context kind-tws-cluster
- kithney nodes hey - kubectl get nodes --context kind-was-cluster

create a folder called miniqube: and install the miniqube

how to change kubectl set context default in kind

kubectl config use-context kind-tws-cluster
kubectl get nodes

- Kubeadm : Agar aapko kubeadm . bare metal aapko agar k8s cluster banana ho

aap ek master banakey multiple nodes ko join kara sakthey hey
ek master node pey kubeadm init hoga jaha pey b kubernates initialize hotha hey vo hojathe hey master node phir aap jithney b nodes chahiye aap join kar sakthey ey - ae jo worker nodes join hothey hey 6443 port number pey join hothey hey
- Now kubeadm has been installed

Aapko ek docker ka container ko chalathey ho ek pod mey - ek pod mey ek sey yah zyada containers chalathey hey

kubernates mey aap autosale , autoheal is cheez chalney key liye deployment chahiye
is deployment ko bahari duniya ko access devoge - isliye service ki zaroori hothi hey jab service banthi hey jabhi as a user aap access karavoge
aekhi cluster pey nginx , mysql b chalra hey toh confussion hotha hoga na kaisey in cheezon ko manage karsakey
jaisa whatsup mey group hotha hey like , family group , friends group isi tarah sey k8s mey b groups hothey hey jismey us group key related jithne b resources hey like pod , deployment , service - us group ko bolthey hey namespcae (ns)

Namespace: ae ek unit hey us particular namespace mey jithney b resources hey us group mey rehthey hey - ek group key resources aur ek group key resources ko disturb nahi karthey , complete isolation jaisa ki aapko manage karnemey easy rahega

cd kind-cluster
kubectl get namespace (or) kubectl get ns - agar aapney koi b name space nahi diyetoh ae by default namespace mey jatha hey
kubectl get pods - jo default ns usmey kuch b pods nahi hey
kubectl get pods -n kube-system - cluster key pods is namespace mey rehthey hey
isliye kubernates mey bohot sarey applications chala sakthey hey alag alag namespace sey
kubectl create ns nginx - to create the namespace
kubectl run nginx --image=nginx → is used to run the pods inside the pod nginx container will be running
galthi sey aapka pod hey default mey create hoga - aap pod ko delete ko b delete hosakta hey
- kubectl delete pod nginx
kubectl run nginx --image=nginx -n nginx
- kubectl get pods -n nginx
- is tarikhey sey pods - namespaces create kar sakthey - aap ae saari cheeze vi yaml banayenge - isko manifest file bolthey hey - aap jo chahthey hey manifest mey banado vahi run hothey hey
kubectl pod delete nginx
kubectl delete ns nginx

namespcace: jaisa humne kind cluster diya is ka b kind namespace , version api ka v1 , ab jo info deni heyna vo rehthi hey metadata object mey rehthi hey , metadata mey name: nginx
- kubectl apply -f namespace.yml - isko apney apiserver pey lekey jav aur create kardo , jab aap apply karthey ho thab aapka object ban jatha hey
pod.yml : kind pod rahega , apiVersion: v1 rahega , aur info metadata mey name: nginx-pod , ae pod kisi group or kisi namespace mey create karna chahtha hun - namespace: nginx iska spec google mey k8s pod - aur aapko yaha mey pod yaml milthey hey , spec mey containers bana sakthey hey usmey containers bana sakthey hey ek aur ek sey zyada usko name banana padtha hey , spec: name: nginx , image : nginx:latest ,ports: - containerPort: 80 - ae banigai manifest pod
- kubectl apply -f pod.yml -→ kubectl get pods -n namespace
- nginx container mey kaisa ghusey -→ kubectl exec -it pod nginx-pod -n nginx -- bash
- curl 127.0.0.1 -→ aapka nginx chalra aap browser mey chalayetoh maza aatha hey
- ae pod dhang sey chalra yah nahi - kubectl describe pod/nginx-pod -n nginx -→ aapkey pod key sath kya kya hua ae sab bata detha hey
- is pod banana khasiyath nahi hey is pod ko scalable banana toh deployment lagega

deployment: kind : deployment , aur apiVersion: apps/v1 - google k8s deployment - ismey aapko manifest rahega - apiVersions differ rehthey hey harrek service key liye apiVersion alag rehtha hey phir metadata rahega , name: nginx-deployment , namespace: nginx -→ spec:
- deployment kiun chahiye - autoheal , scalable karna hey thab deployment ki jaruri rehthi hey replicas - ae replica banakey dega Replicaset/statefulset/deploymentset
  1. deployment: Replication controller - ek pod ko pura traffic nahi dey sakthey vo crash hojayega -aap uska replica banathey hey - replica controller pod key replicas ko manage kartha hey like clone

deployment aur replica same rehtha hey - ae aapko ek feauture detha hey rollingUpdate - without downtime rolling update one by one it makes down the pod and update the pod
ae jo mera pod hey - is pod mey jo nginx chalra heyna uskey 2 replica chahiye - toh deployment ko pod pehchan mey aana chahiye -isiliye is pod ko label dena chahiye , us pod ko select karnekey liye selectors b chahiye - pod ka label and deployment ka selector criteria match hojai toh usko replica kardo
kubectl apply -f deployment.yml
kubectl delete -f pod.yml
kubectl scale deploymeny/nginx-deployment -n nginx —replicas=5
kubectl scale deploymeny/nginx-deployment -n nginx —replicas=1
kubectl set image deployment/nginx-deployment -n nginx nginx=nginx:1.27.1
kubectl rollout deployment/nginx-deployment -n nginx set image=nginx:1.27.3

rolling update karthey karthey error b aagaya aapkey saare pods pey error nahi aayega kuch pods run horey hongey kuch pods update horey honge sequenctially
rollingupdate kiunn karthey hey suppose merko v1 key pods hey - isko latest version v2 sey update karna hey thab deployment without downtime isko rollback kardega , replicaset mey na sab kuch delete kar detha phir update kartha hey
kubectl delete -f deployment.yml
cp deployment.yml replicaset.yml
- 2. ReplicaSet : agar aapney bola hey ae pod ki temple image use karni hey nginx ki replica honi hey 4 , 4 pods banjayenge lekin ismey jab roling update mey downtime aatha hey -
kubectl apply -f replicaset.yml
replica mey rollout -→ roleback nahi karsakthey
kubectl get replicasets -n nginx
replicaset using the same deployment technology - deployment is far advanced .
- 1. Statefulset : har ek pod kabhi bhi create / delete hosakthi hey satateful set kya bolthi hey harrek pod ko numbering dedunga 1 , 2 ,3 4 - har ek pod ki state by number sequence mey maintain rahegi
- 4..daemonset: ae jo pods hey -n nginx ae kisko miley hey aapkey pass 3 workers hey
kubectl get pods -o wide -→ ae pods kispey assign hue - worker 3 mey nahi mil paye
deamon sets ensure karthey hey jithney b aapkey node hey atleast usmey ek pod toh chalthey rahey -
kubectl apply -f daemonset.yml
kubectl get pods -n nginx
kuectl get pods -o wide - daemonset worker1 pey ek pod , worker2 pey 2 , worker3 - 1 ensure kartha hey .
- saare ek kaam hi karthey hey - relication control hi karthey hey
daemonset ka kaam hotha hey har ek node mey atleast ek (replica)pod run kartha hey
replicaset - jithey b aap boley uthey replicas maintain kartha hey
satefulset : ka kaam jithey replica ho vo apney sath ek state maintain karthey chale
deployment- jithne b replica hey , rolling update , scaling , auto healing karthey rahey

4.Jobs& CronJobs

- jobs: aapko ek container sey ek single task karana hey - aap nahi chalthey vo cheez as a sever chalthi rahey forex: tws as a sever chaltha hi rehtha hey , aap chahthey ho ek hi job hey vo chaley aur khatham hojaye
  - like it can be comething like - paching , system update , backup ,, or creating directory
  - vim job.yml → ab jo aap banaraheho vo single chalega yah batches mey chalega - like single thread
    - aaya backup liya khatham
    - or ae jo task hey parallel li chalado three pods same task pey toot padey hey
    - spec important completism: 1 parralelism: 1 agar 2 rakhdunga toh same task 2 pods create hokey job run hongey
    - busybox ek image rehtha hey is ka kaam sirf commands chalane ka rehtha hey
    - kubectl apply -f jobs.yml
    - kubectl get pods -n nginx → status completed → ae job chala vusney 10s hello dst print kiya aur khatham kiya
    - pod ki b life cycle rehtha hey - state kya rehtha hey - sabsey pehle container creating , running , terminating , phir completed, imgpullbackoff, errpullimg
    - kubectl logs pod/demo-job-qqrkh -n nginx
    - mai job ko phirsey run karsakta hun
    - kubectl delete -f jobs.yml → job ko delete karkey phirsey create karna padega
  - Cron: ae jo task hey ae particular schedule pey chale toh is job ko bola jatha hey cron job
    - a task that you can schedule ki vo pod chaltha rahega create hotha rahega chaltha rahega but complete tabhi hoga tha vo scedule thabhi complete hoga
    - vi cron.yml
    - volumes - ae aapka pod hey uskey andar ek container chalra hey - ae aapka host hey device ae container ae device iskey beech mey data ko persist karana chathey ho kiunki data loss na hojay toh isliye volume bana dethey ho aur us volume ko container sey mount kara dethey hey
      - kubectl get pods -n nginx
      - kubectl logds pod/minute-backup -n nginx

cd nginx 
1. vi namespace.yml

kind: Namespace
apiVersion: v1
metadata:
  name: nginx

2. vim pod.yml 

kind: Pod
apiVersion: v1
metadata:
  name: nginx-pod
  namespace: nginx
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80

3. deployment.yml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-deployment
  namespace: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      name: nginx-dep-pod
      labels:
        app: nginx

    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

4. replicaset 
kind: ReplicaSet
apiVersion: apps/v1
metadata:
  name: nginx-replicasets
  namespace: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      name: nginx-rep-pod
      labels:
        app: nginx

    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

5. daemonset.yml

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: nginx-daemonsets
  namespace: nginx
spec:
  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      name: nginx-dmn-pod
      labels:
        app: nginx

    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

5. job.yml 
  kind: Job
apiVersion: batch/v1
metadata:
  name: demo-job
  namespace: nginx
spec:
  completions: 1
  parallelism: 1
  template:
    metadata:
      name: demo-job-pod
      labels:
        app: batch-task
    spec:
      containers:
      - name: batch-container
        image: busybox:latest
        command: ["sh", "-c" ,"echo Hello Dosto! && sleep 10"]
      restartPolicy: Never

vi cron.yml 

kind: CronJob
apiVersion: batch/v1
metadata:
  name: minute-backup
  namespace: nginx

spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        metadata:
          name: minute-backup
          labels:
            app: minute-backup

        spec:
          containers:
          - name: backup-container
            image: busybox
            command:
            - sh
            - -c
            - >
              echo "Backup Started" ;
              mkdir -p /backups &&
              mkdir -p /demo-data &&
              cp -r /demo-data /backups &&
              echo "Backup Completed" ;
            volumeMounts:
              - name: data-volume
                mountPath: /demo-data
              - name: backup-volume
                mountPath: /backups
          restartPolicy: OnFailure
          volumes:
            - name: data-volume
              hostPath:
                path: /demo-data
                type: DirectoryOrCreate
            - name: backup-volume
              hostPath:
                path: /backups
                type: DirectoryOrCreate

STORAGE:

jab hum logoney cron job banyatha ae volumes aur volume mounts kuch toh kiya
samjo jab b aap ek pod banathey ho - vo pod aek server aur ek node pey chalra hotha right , ae pod kabhi bhi delte or destry kuch b ho saktha hey
is pod key andar ka jo b data hoga uska kya karoge agar vo delete hojayetoh
replication boltha hey desired state auto heal artha hey vo jo containers delete hua vo data b chalgaya
aapko aapkey pods ka data ko persist karna padtha hey - persist karaneykey liye aapko host key sath bind (persist) karna padtha hey
us cheez key liye persistent volume and persistent volume claim zaroori hothi hey ae hey kya
aisa samjo ae aapka host machine hey ec2 or docker iskey andar kuch storage hey in 30 gb mey sey kuch kuch section allocate karna chahtha hun 1GB allocate karna chahtha hun mainey 1GB ka persistent volume banaya
vim persistentvolu.yml - mainey kind: PersistentVolume liya , apiVersion:v1 liya aur metadata : name:local-v , labels: app: local ismey mainey spec diya - ae p.v iska capacity kithna chahiye - storage :1Gi; accessModes: ReadWriteOnce , mai chahtha hun ki agar mainey delete b kardiya ae jo volume hey host machine mey ae retain kardo → persistentVolumeReclainPolicy: Retain ; storageClassName: local-storage ; hostPath: path: /mnt/data
mai nahi chatha ae sirf aur sirf nginx namespcae may aajaye
kubectl -f apply persistentvol.yml - ae banahua hey available hey isko claim karna padega

vi persistentvolume.yml

kind: PersistentVolume
apiVersion: v1
metadata:
  name: local-pv
  namespace: nginx
  labels:
    app: local
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  hostPath:
    path: /mnt/data

aap ae sipper lekey aagaye kiska hey pata nahi lets say mera hey agar mai claim karunga toh vo mera hey
sirf volume banadiye available hey isko claim karna padtha hey

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-pvc
  namespace: nginx
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: local-storage

kubectl get ns
kubectl get pv
kubectl apply -f pvclaim.yml

mai ek gb request karrahun mai apney persistent volume sey 1gb request karra hun

ab jo pv available tha vo bound hojaya hey pvclain sey , ab ae jo pod mey data delete ho jaratha ab mai kya chhatha hun ae mera pod host machine mey /mnt/data mey apna data bachakey rakhey - toh mainey 1gb ka pv banaya aur us 1gb ka claim banaya ab mai is claim ko ae mai is pod ko dedunga

kubectl get deployment -n nginx -→ kubectl delete deploymentyml

mujehy aisi deployment banana hey jiska data delte na ho kaisa banaunga ae jo container hey usko mai olunga volumeMounts iskey andar jo nginx ka data mount path : /var/www/html vo mount rahey name: my-volume key sath mount rahey ab jo meri volumes hey container key bahar rehtha name hey : my-volume

is ka jo volume persistentvolumeclaim: claimName: local-pvc

kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-deployment
  namespace: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      name: nginx-dep-pod
      labels:
        app: nginx

    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 200m
            memory: 256Mi
        volumeMounts:
          - mountPath: /usr/share/nginx/html
            name: my-volume
      volumes:
        - name: my-volume
          persistentVolumeClaim:
            claimName: local-p
vc

kubectl apply -f deployment.yml
kubectl get pv -n nginx
kubectl decribe pod/pod-name -n nginx
kubectl get pods -n nginx -o wide
kubectl get nodes -ae node hey cont
docker ps → docker exec -it cont_id bash → cd /mnt/data/ ls - mainey kuc create nahi kiya

ae deployment bahari duniya ko dikhani hey toh iskey liye service ki zarori hothi hey service lagtha hey - service redirect kar detha hey

ae service bohot saare deployments mey redirect kara dethi hey

service: kind service rahega , api version rahega v1 aur metadata rahega name and namespace aur spec mey vahapey selctors dena padega aapko ae service kiskey liye expose karni hey , jiska b label app:nginx hey uska expose karunga
kaha pey karunga tcp protocal - port - baharki duniya mey kounsi port sey expose karna chahthey ho - targetPort :cont port hey

services key alag alag types rehtey hey

type: clutser ip , node-port , loadbalancer , external ip , headless service

agar mai cluster ip kar detha hun aapki cluster hey vo nikal key aatha hey aur uskey andar ek port assign ho jatha hey aur usko aap port map kar sakthey
agar mai nodeport likhdun - aapkey purey node ka port likalna padtha hey like 30000 sey bathis hazzar thak
loadbalancer generally cloud services key liye use karthey hey
aur external ip rehthi hey vo ek external ip usmey aapkp koi aur ip address static ip dalkey laga sakthey hey
headless: generally stateful sets key sath use hothi hey

vim service.yml

kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  namespace: nginx
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: ClusterIP

mera nginx service bahar ki duniya ka port number 80 aur andar ka cluster ka port pod ka port number sey map kardega , for anything that has app:nginx
kubectl apply -f service.yml
agar mai apney browser pey jau - ip :80 nahi access karpayega actually chalna chahiye
nahi chalra kiunki - aapka cluster hey vo ek docker container hey , toh docker container ka port forward karna padtha hey , address ip address expose karna padtha hey
- sudo -E kubectl port-forward service/nginx-service -n nginx 80:80 --address=0.0.0.0
- ab access kardo - aek chotasa example lelethey hey - laundheshubham notes-app dev branch - isko mai kubernates project banani hey
- git clone https://github.com/LondheShubham153/django-notes-app.git
  - pehle docker image banaletha hun - > docker build -t notes-app-k8s .
- ab ae jo meri image hey ae locally nahi chalney vali isko dockerhub mey pahunchana chahiye isliye docker hub ka login chahiye isliye mai dockerhub mey login karlunga - toh mai docker hub mey token bana detha hun
- docker image tag notes-app-k8s:latest vaseem143/notes-app-k8s:latest
- docker push vaseem143/notes-app-k8s:latest
- cd k8s → vim deployment.yml
```
  kind: Deployment
  apiVersion: apps/v1
  metadata:
    name: notes-app-deployment
    namespace: notes-app
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: notes-app

    template:
      metadata:
        name: notes-app
        labels:
          app: notes-app

      spec:
        containers:
        - name: notes-app
          image: vaseem143/notes-app-k8s:latest
          ports:
          - containerPort: 8000

  vi service.yml

  kind: Service
  apiVersion: v1
  metadata:
    name: notes-app-service
    namespace: notes-app
  spec:
    selector:
      app: notes-app
    ports:
      - protocol: TCP
        port: 8000
        targetPort: 8000
    type: ClusterIP
```
  - kubectl get pods -n notes-app
  - kubectl port-forward service/notes-app-service -n notes-app 8000:8000 --address=0.0.0.0

aisa samja agar mai /app - ae app khuley agar mai /nginx karey nginx khuley - toh aisa toh aisa khuley

mai chahtha hun ki redirect karey services ko b

Ingress

iskey liye aapko ek cheez use karni padthi hey ingress - ingress aapka traffic re -routing kartha hey is cheex ko bolthey hey ingress

ek high level diagram sey dekhetoh ae aapka cluster hey ismey bohot saare pods hey - ae pods manage horey ek deploymen sey - aur is deployment ko access karnekey liye ek service ki zaroorath padthi hey iskey liye aap service create kardiye aisa hi alag deployment service aur 2 hey
multiple services ko kaisa route karna hey iskey liye ingress use hotha hey

toh ingress jo hotha hey basically ek service hothi hey it allows us to manage service and traffic
agar mai /nginx/ karegetoh nginx jana agar mai /app kareth notes appp redirect honi hey in saarey cheezkon ko mai ek hi namespace mey chalana chahtha hun
service ko b - so delete kardo deployment and svc ko edit karkey namespace phir sey create karo
mere pass ninx ki and notes-app ki service b hey - mai ae jo daigram banana chahtha hey vo bangayi chalo isko redirect kar wathey hey
aap jo ingress karrehono isko ingress conroller lagtha hey jo aapko cluster level pey routing karwaney mey help kartha hey - nginx controller aur b controllers hey
k8s ingress controller - load balancing and redirecting mey b help kartha hey
aapko kind cluster document mey jakey ingress click karo
cd /home/ubuntu/kubernates-in-one-short - ab mai kubectl apply -f nginx-ingress apply karunga -→ kubectl apply -f https://kind.sigs.k8s.io/examples/ingress/deploy-ingress-nginx - ek ingress controllers services banaey honge
kubectl get ns
kubectl get pods -n ingress-nginx
similarly aapko services b karni hey get → kubectl get svc - isi service ko badh mey expose karna padega
ab mai apney nginx mey jakey ek nayi file banaunga - ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-notes-ingress 
  namespace: nginx 
  annotations:
       nginx.ingress.kubernates.io/rewrite-target: / # usmey jo / hey usko rewrite karna hey rewrite 
               # rewrite karkey /nginx kardo 
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /nginx # jab b ae path nginx ai redirect kardo 
        backend:
          service:
            name: nginx-service # redirect kardo nginx-service per 
            port:
              number: 80 # service ka port hey 
      - pathType: Prefix
        path: / # jab b mai bolum / app redirect to notes-app-service
        backend:
          service:
            name: notes-app-service
            port:
              number: 8000

kubectl apply -f ingress.yml
kubectl get ing -n nginx
kubectl get all -n nginx -→ ae jo ingress hey ae ek ingress controller key sath chalra hey jo ingress nginx name ka controller hey jo kind cluster mey install kiye usko expose karna hey
kubectl get svc -n ingress-nginx
sudo -E kubectl port-forward service/ingress-nginx-controller -n ingress-nginx 8080:80 --address=0.0.0.0
ab mai ip:8080 - ismey nginx chalra aur / pey aur /nginx - nginx b port 80 i mean / mey chalega agar mai /nginx sey karun mera nginx open huey isliye annotations matlab extra information
ingress ki specifications ko customize kar sakthey ho
kubectl apply -f ingress.yml
sudo -E kubectl port-forward service/ingress-nginx-controller -n ingress-nginx 8080:80 --address=0.0.0.0
kubectl delete -f namespace.yml - sab jo b nginx namespace key andar hey sab delete hojathey hey

Statefulset:

mkdir mysql

jab b hamare koi applications hothey hey django , flask , aur springboot apps ae saarey applications ,agar aapkey state maintain nahi hothi hey toh b chaltha hey . UI hey user kabhi use karega band karega → ae basically stateless hothey hey
is ka jo data rahega mysql , mongodb ae saarey db hey vo stateful rehthey hey
mysql ka data hey vo always persisted hona chahite - timeley usmey transactions matlab read write ae sab ek dum sahi sey hona chahiye state key aap kabhi bhi khilwat nahi kar sakthey when it comes to databases
jab b aap stateless applications hothey hey generally app use karthey hey
- deployment , replicaset , daemonsets etc
jab appki stateful applications like mongodb , mysql - use karthey hey stateful sets ismey b replica rehtha hey - ae jo pods banthey hey ae pod apney sath ek state carry karthey hey like 1, 2 ,3 ,4,5 sequence mey numberd hothey hey

vi statefulset.yml

kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: mysql-statefulset
  namespace: mysql

spec:
  serviceName: mysql-service
  replicas: 3
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.0
        ports:
        - containerPort: 3306
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 200m
            memory: 25Mi
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: MYSQL_ROOT_PASSWORD

        - name: MYSQL_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mysql-config-map
              key: MYSQL_DATABASE 
        volumeMounts:
        - name: mysql-data
          mountPath: /var/lib/mysql

  volumeClaimTemplates:
  - metadata:
      name: mysql-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

pehle ek namespace mysql bana dethye hey phir statefulset.yml
mai is valey statefulset ko ek spec dedunga ismey pehle replica:3 , mai lables and selectors use karunga - selector: matchLabels: app: mysql

important part template mey aatha hey
jo mysql ka container rehtha hey vo always environmnet variables mangtha hey
env : mysql key jo environment dalna hey vo kaisa dalthey hey → k8s stateful environmnet google it
but agar aapko ek mysql ka stateful set banana hey is pod ko jo storage chahiye volume chahiye vo aapko stateful set mey hi dalna padega
mai yaha aek volume banaunga aue volume claim template b ismey hi karunga
jaisa mainey statefulset replica hey uska pod ka ek service b honi chahiye - iski service ki ek khasiyath rehthi hey - headless service - iska sab same rahega
```
              kind: Service
              apiVersion: v1
              metadata:
                name: mysql-service
                namespace: mysql
              spec:
                clusterIP: None
                selector:
                  app: mysql
                ports:
                - name: mysql
                  protocol: TCP
                  port: 3306
                  targetPort: 3306
```
- ae service internally use key liye kahi expose nahi hothi , bahari duniya key liye acccess naho honi chahiye isliye agar mere app ko data chahiye thorough service it can able to access
- ae jo service sirf aur sirf statefulset key sath bind rahey isliye → statefulset.yml mey jakey bind kardo
- kubectl apply -f service.yml
- kubectl apply -f statefulset.yml
- kubectl get pods -n mysql
- kubectl get svc -n mysql
- kubectl exec -it mysql-statefulset-0 -n mysql -- bash
- mysql -u root -p = root
- mainey ek stateful set key sath mysql ki deployment bandiya aur vo b scalable hey
  - show databases;
  - kubectl delete pod mysql-statefulset-0 -n mysql
  - kubectl get pods -n mysql -→ jab b hum delte karey with the same pod name it creates a new one , that is how k8s maintains the state
  - pod humesha koi aur create karthey na charoka name random honge , koi b delet kardo koi naya name banjayega but is case mey aisa nahi hotha , is case mey vahi name ka pod delete kiya vahi name sey new vala bantha
  - iski service, iski volume template , variables , sab tightly coupled rehtha hey , isiliye jab b naya pod bantha hey vo hamesha state ko sath lekey chaltha hey , isiliye known as stateful set
  - deployment mey jo b pods create banthey hey alag name sey banthey hey .

ConfigMaps: kaisey aap aapney pod mey jo b variables hey jo b aapka data hey kaisey ek dam secure tarikheysey pauncha sakthey hey chalo dekhthey hey

aisa samjo ki aapney jo create kiye na statefulset banaya ismey ae jo mysql dataabase ki value dalina kal ko agar aapko change karni padegi aapko stateful sey mey aakey change karna padegi

ki mai chahtha hun ki jithey b meri environmenyt variables , varibales vo alag ek file mey rahey , jo merey conf rehthey hey vo ek alag file mey rahey

kind: ConfigMap
apiVersion: v1
metadata:
  name: mysql-config-map
  namespace: mysql
data:
  MYSQL_DATABASE: devops

- isko aap use karoge stateful set
vim statefulset - jo mera env mysql_Data base hey vo value mai direct nahi liunga liunga valueFrom : name: mysql-config-map ; key: MYSQL_DATABASE; ae as a configMapKeyRef karkey dalna padega
aapki jo config map hey usmey jo aap key dali hui hey key ref sey
kubectl apply -f statefulset.yml

kubectl get statefulset -n mysql

kubectl delete statefulset -n mysql

kubectl apply -f statefulset.yml

kubectl get pods -n mysql

jaisey aapney config maps mey normal plane text valey key:value pair mey data dala usi prakar mey aap agar encode karkey dalna chahthey ho , generally passwords ko encode karna chahthey ho toh secrets mey dalthey hey

Secrets:

vim secret.yml - ismey jo b password hey na usko base64 encode karkey dalna padega

echo “root” | base64

apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
  namespace: mysql
data:
  MYSQL_ROOT_PASSWORD: cm9vdAo=  # base64 encoded for "root"

kubectl apply -f statefulset.yml

kubectl get pods -n mysql

ResourceQuota & Limits : ek pod hey ae jo aapka pod aapkey host machine mey chalra hotha hoga - ab is machine ka kuch configuration hoga t2.medium like 4gbram 2cpu

lets say ae pod hey nginx - pod pey aagaya traffic badthey jara jara aur jheltha jara 4gb ki ram ae akela pod hi khagaya
lets say aur ek pod aagya agar nginx pod hi khagaya toh 4gb toh ae bichara dusra pod kya karega isko kuch b nai milega
jab b aap apney pods banathey hey - aap usmey kithna resources aapko chahiye , maximum kithna ley saktha hey pod ae aapko dena padega decide karna padega
meri deployment file hey us file mey humko limit deni hothi hey

            resources:
                      requests:
                        cpu: 100m
                        memory: 128Mi
                      limits:
                        cpu: 200m
                        memory: 256Mi

ae toh minimum request , max 2 roti sey zyada nahi chahiye uske liye limits use karthey hey
issey na aapka pod ithna hi resource leyega -→ aap isko create karkey describe kardo

Probes:

kaam chota sa hi rehtha hey - probes aisa samjo ki vo 3 type key hothey hey
1. liveness Probe
2. readiness probe
3. startup probe
probe matlab ek request rehthi hey ki ae aapka pod dhang sey kaam karrahey yah nahi - lets say ae pod pey 8000 pey chalra but aapko make sure karna hey ki vo 8000 mey hi chalrahey , agar aapka pod zinda hey yah nahi aapko live hey yah nahi liveness probe test kartha hey
agar aapka pod create hotha hey yah ready ho gaya thab readiness probe test kartha hey
startup probe - jab aapka pod start hora hey startup probe key sath request check kartha hey

livenessProbe:
          httpGet:
            path: /
            port: 8000

is pod mey mera readiness aur liveness probe trigger hua yah nahi
- kubectl describe pod podname -n nginx
- probe just check kartha hey internally - yaha pey intervals b dal sakthey hey initial delay ae sab

Taints & Tolerants:

toh is valey diagrams ko dekhthey hey , mai jabbi apney cluster ko boltha hun kubectl apply -f deploy - api- server scheduler ko boltha hey bhai pod schedule kardo , schedular apney hisab sey dekhta hey pod woker 1 , 2 ,3 kounsey node mey bhejna hey ae schedular decide kartha hey
taint , tont - hum tont marthey hey na , uskey ghar ko nahi jathey na agar ek relative humko tont kartha hi rehthey hetoh
vaisa hi aap schedular ko cahthey hey ki node1 pey pod schedule na karey toh aap node1 ko taint kardogey
taint is a way of telling your k8s cluster ki is particular node mey aap pod nahi schedule karey
kuch kuch relatives taunt tont marthey hey lekin hum ghar chale jathey hey sehlethey hey tolerate
aapki koi tainted server hey lekin aapko chalana chahthey ho usko aap tolerants dal sakthey ho
hum kaisa taint karwana hey

kubectl get ns → cd nginx -→ cat pod.yml - ae aek pod hey ae jo pod hey agar mai chalaunga ae kidar jayega agar mai worker node ko taint kiya toh ae pod us node pey run nahi hoga

kubectl taint node tws-cluster-worker prod=true:NoSchedule
kubectl taint node tws-cluster-worker2 prod=true:NoSchedule
kubectl taint node tws-cluster-worker3 prod=true:NoSchedule
kubectl apply -f namespace.yml
kubectl apply -f pod.yml
kubectl get pods -n nginx - ae pending mey hey kiun matlab , hamare 3 ko 3 nodes tainted hey , jo b taint ho jatha hey usmey schedule nahi kar patha .
ae pod bechara ko ek node dey dethey - kubectl taint node tws-cluster-worker3 prod=true:NoSchedule- - hyphen matlab untainted
Tolerants:
tainted pod mey b mai kaisey chalaunga - pod key spec mey jakey - tolerations: - key: “prod” operator: “Equal” value: “true” - aisa koi b cheeze taint pada hey jismey pod=true and noschedule dala hey tolerate karkey chalado karkey bolre

tolerations:
    - key: "prod"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"

HPA & VPA: Horizantal pod autoscaling & vertical pod autoscaling :

sabsey pehley samajthey hey autoscaling - autoscaling ab har koi microservice architecture mey aagaye ab sabko based o traffic autoscale karna chahthey hey
jab autoscale ki bath aathi hey 3 types hey
- HPA: matlab agar aapka jo pod hey mast traffic aagaya ae jo pod hey apney aap scale hojayega like replica ban jathey hey , replica ko badathey ho number badathey hey - normal applications use kar sakthey
- VPA: pehley aapka pod aisa tha vo maxium resources 100mb use kartah hey max ae 200mb ka traffic jhel saktha hey - agar isko autoscale karna chahthey ho ae banjayega hulk ae pod ki thakath ko badathey ho , resource and limits ko badathey hey - basically ae stateful applications ko use karthey hey
- KEDA - kubernates event driven autoscaling : vo dekhtha hey aapka event kaisa hey , bohot zyada traffic or rerouting hey toh vo either hpa pey lagadeyega bohot zyada overload or stress aara hey ae vpa mey laga dega - based on the nature of your metrics or events keda hpa or vpa sey selct karletha
- metrics kya hotha hey - countafiable resources key numbers like cpu and ram kithna use hora hey
- kubectl get nodes
- ae jo node hey kithna traffic aara hey , kithna kya hora hey iski metrics aapko pata honi chahiye iski metrics
- kubectl top node → issey metrics nahi hey - kubesystem mey metrics severs ,hona chahiye and vo metrics server aapkey node mey aapkey pod mey kithne resources , cpu kya scena hey kithney use horey ae sab metrics detha hey
- miniqube hey toh usmey karlenge - miniqube addons enable metrics-server- ae karkey badiya chala sakthey hey
- kind cluser mey metrics chahiye - toh aap github mey milega - usmey copy karkey idhar run karo -→ github hpa and vpa ey javo
If you are using a Kind cluster install Metrics Server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Edit the Metrics Server Deployment

kubectl -n kube-system edit deployment metrics-server

Add the security bypass to deployment under container.args

- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP,Hostname,ExternalIP

Restart the deployment

kubectl -n kube-system rollout restart deployment metrics-server

Verify if the metrics server is running

kubectl get pods -n kube-system
kubectl top nodes
kubectl top pod -n nginx

jaisey humney coreconcepts key nginx , pv pvc key liye statefulsets mysql , django notes app aek application dekhey

ab hpa and autoscaling key liye ek naya dekhenge apache - ae b ek httpd server hey
ek namespace banana hey - kubectl create namespace apache , ab mujhey is apache ka deployment banana hey us deployment ko hpa sey automatically scale kar wana hey
Steps to implement HPA:
- Update the Deployments:
  - We'll modify the existing Apache deployment YAML files to include resource requests and limits. This is required for HPA to monitor CPU usage.

    vi apache-deployment.yaml

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: apache-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: apache
      template:
        metadata:
          labels:
            app: apache
        spec:
          containers:
          - name: apache
            image: httpd:2.4
            ports:
            - containerPort: 80
            resources:
              requests:
                cpu: 100m
                memory: 128Mi
              limits:
                cpu: 200m
                memory: 256Mi

sabsey important containers ki resources quota and limit set karni hey

    vi service.yml 

    apiVersion: v1
    kind: Service
    metadata:
      name: apache-service
    spec:
      selector:
        app: apache
      ports:
        - protocol: TCP
          port: 80
          targetPort: 80
      type: ClusterIP

kubectl get all -n apache
sudo -E kubectl port-forward svc/apache-service 8081:80 --address 0.0.0.0
ae application ko public mey daldethey toh sab log access karengetoh mera cluster ruk saktha hey
mai chahtha hun - kubectl scale deployment apache-deployment -n apache —replicas=3 -→ ae manuall hey - hum dekha samja - ae scalling hey autoscalling nahi - autoscaling key liye hpa banana hey

            #apache-hpa.yaml
            apiVersion: autoscaling/v2
            kind: HorizontalPodAutoscaler
            metadata:
              name: apache-hpa
              namespace: apache
            spec:
              scaleTargetRef:
                apiVersion: apps/v1
                kind: Deployment
                name: apache-deployment
              minReplicas: 1
              maxReplicas: 5
              metrics:
              - type: Resource
                resource:
                  name: cpu
                  target:
                    type: Utilization
                    averageUtilization: 20

- kubectl get pods -n apache

kubectl apply -f hpa.yml
kubectl get hpa -n apache
ab ae jo url hey it works - isko load kaisa diyenge load bananey ka tarikha
aek pod banayenge - loadgenerator karkey
kubectl run -it load-generator —image=busybox -n apache — bash

Stress Testing

To see HPA in action, you can perform a stress test on your deployments. Here is an example of how to generate load on the Apache deployment using 'BusyBox':

    kubectl run -i --tty load-generator --image=busybox /bin/sh

Inside the container, use 'wget' to generate load:

    while true; do wget -q -O- http://apache-service.default.svc.cluster.local; done

This will generate continuous load on the Apache service, causing the HPA to scale up the number of pods.

Now to check if HPA worked or not, open a same new terminal and run the following command

    kubectl get hpa -n apache 
    kubectl get pods -n apache

Note: Wait for few minutes to get the status reflected.

agar mainey rok diya → watch kubectl get pods -n apache

kubectl get hba -n apache -→ humko samaj aagaya how HPA automatically scale the application same notes-app mey deployment mey resource and limit use karo and just change name deployment name in HPA.yml
kubectl apply -f deployment.yml
kubectl get pods -n ninx
kubectl apply -f service.yml
kubectl apply -f hpa.yml
kubectl get hpa -n nginx
kubectl get pods -n nginx
kubectl delete ns nginx .

Vertical pod autoscaler key files ko download karna padtha hey

For VPA

git clone https://github.com/kubernetes/autoscaler.git
cd autoscaler/vertical-pod-autoscaler
./hack/vpa-up.sh  # jo jo prerequisite chahiye vpa key liye vo install karthi hey

Verify the pods on VPA

            kubectl get pods -n kube-system

  vim vpa.yml 

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: apache-vpa
  namespace: apache
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: apache-deployment
  updatePolicy: # idhar aap update mode dalthey hey 
    updateMode: "Auto" # Options: "Off", "Initial", "Auto"

kubectl get pods -n apache
kubectl get hpa -n apache
kubectl delete -f hpa.yml
kubectl apply -f vpa.yml
kubectl get vpa -n apache
take another terminal → kubectl get svc -n apache
kubectl get pods -n apache
kubectl get vpa -n apache
kubectl run -i -tty load-genera
tor —image=busubox -n apache /bin/bash
while true; do wget -1 -O https://apache-service.apache.svc.cluster.local ; done
sudo -E kubectl port-forward svc/apache-service 8081:80 --address 0.0.0.0
kubectl get vpa -n apache
kubectl top pod -n apache —> this is how you have enlarged your pod resource
kubectl describe pod podname -n apache
kubectl top node
kubectl get pods -n apache -o wide
k8s node affinity

Node Affinity / node selectors : agar aapkey pod ko aapkey node pey schedule karana hey toh us node key bharemey thodasa batado like node name , type , key , resources agar usko match karthey hue milgaya

node affinity is opposite to taint and tolerants

Cluster Administration :

RBAC( Role based Acess control):

ae aapka ghar , aap ek ghar mey ho bolnekey liye aapka , lekin papa mummy ka haq hotha hey aap vaha key naukar b hothey hey - aapkey gharmey ek role hotha hey right ,
like kya kya cheeze karne wale - 1. breadearner , 2. house wife
lets say aek role hey mai sabji lekey aaney wala → hamare pass ek user hey shubham ab is shubham ko ae role dedetha hun sabji lekey aana , ae user shubham ae aur ae role ek dusre sey kaisa bath karthey → through role binding
jab b aap rbac padthey ho ithna samjo kubernates key resources hothey hey uskey access aapka hey yah nahi uskey liye ek role bantha hey
home = k8s cluster - us cluster key andar kya kya kar sakthey hey kya kya nai kar sakthey hey us cheez ko rbac bola jatha hey
rbac padney key liye do cheeze padni padthi hey
1. service account : service account aapka aek temporary user jaisa hi rehtha hey, us service account mey as a user kya karthey hey lets say mai as a user ek anager hun ek manager name ka ek service account ho saktha hey - treated as a user only
  - ae namespace level pey b ho saktha hey
  - namespace level pey hotah hey role - kya deployment ko get , update , delete kar saktah hun - us service account ko is role ko denekey liye role binding
  - usi prakar cluser level pey role ni hothey us level pey hothey hey cluster role - us cluster role ko bind karnekey liye cluster role binding hotah hey
2. user : user aapka pura cluster level pey hothey hey - pure key pure cluster level pey
toh jab aap rbac padthey hey ek simple basha mey matlab hotah ehy aapka user or service account ko kithna access dey sakthey ho dena chahthey ho , kithna uska access ko controll kar sakthey hey

2 level pey kar sakthey hey - yah toh namespace level pey resources ko daldo or cluster level pey dal sakthey hey

kubectl get nodes
cd apache → kubectl get ns → kubectl get pods -n apache → kubectl delete -f . →
jab b aap rbac samajthey ho - kubectl auth whoami→ current user info we will get
kubectl auth can-i get pods
kubectl apply -f namespace.yml
kubectl auth can-i get pods -n apache # iam an admin so mai kisi b namespace mey ghuskey kuch b kar saktha hun
kabhi kabhi aapko alag alag apps banana padtha hey , aap nahi chavoge ki har ek user us application ko access kardo .
i dont want to give admin user to fresher what i he deltes the files so i will create a role
Create Roles and RoleBindings:
- Role for Managing Apache Deployment.
  We'll create a role that allows managing Apache resources within the apache-namespace.

# apache-role.yaml
apiVersion: rbac.authorization.k8s.io/v1  # ae group agar apiGroups mey * devige th sab access 
kind: Role
metadata:
  namespace: apache-namespace
  name: apache
rules:
- apiGroups: ["", "apps", "extensions"]
  resources: ["deployments", "services", "pods"]
  verbs: ["get", "apply","list", "watch", "create", "update", "patch", "delete"]

kubectl apply —f role.yml

kubectl get role -n apache # is role ko agar mujhey kisiko bind karna hoga toh mai isko bind kardunga

but uskey pehley mujhey service account chahiye - > vim service-account.yml

Create Users (Optional):

If you are using a Kubernetes distribution that supports user management, you can create users and assign them the respective roles. Here, we'll use basic ServiceAccounts for simplicity.

#apache-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: apache-user
  namespace: apache

kubectl apply -f service-account.yml
kubectl get serviceaccount -n apache # kya mera apache user kya mera deployment ko get kar saktha hey
kubectl auth can-i get pods -n apache —as=apache-user # agar mai as a apache-user mai get kar saktha hun kya
just ab thak ek serviceaccount user create hua , aur role create hua but role binding nahi hua
RoleBinding for Apache Manager We'll bind the role to a user or service account.

# apache-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: apache-manager-binding
  namespace: apache
subjects:
- kind: User
  name: apache-user # This should be replaced with the actual user name
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: apache-manager
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f role-binding.yml

kubectl get rolebinding -n apache

kubectl auth can-i get pods —as=apache-user -n apache

Verify Roles and RoleBinding

To cross-verify the RBAC setup, you can perform a series of checks and actions to ensure that the roles and permissions are applied correctly. Here’s how you can do it:

Check Roles:

kubectl get roles -n apache-namespace

Ensure apache-manager and nginx-manager roles are listed.

Check RoleBindings:

kubectl get rolebindings -n apache-namespace

Ensure apache-manager-binding and nginx-manager-binding role bindings are listed.

Verify ServiceAccounts

Check ServiceAccounts:

kubectl get serviceaccounts -n apache-namespace

Ensure apache-user and nginx-user service accounts are listed.

Test Access Using Impersonation

You can use the 'kubectl auth can-i' command to verify the permissions granted by the roles.

Test Apache User Permissions:

kubectl auth can-i create deployment -n apache-namespace --as=apache-user
kubectl auth can-i get pods --as=apache-user -n apache 
kubectl auth can-i delete service -n apache-namespace --as=apache-user

# role base access control

jab aapko monitoring , logging us level key cheeze karni hothi hey thab aapko cluster role aur cluster role binding zaroori hothi hey

ab humlog dekhney valaey hey - rbac mey cluster level pey kaam kaisey hotha hey

cluster level pey kaam karna hey vo kiun karna hey
lets say mai purey cluster key metrics , monitor , logs dekhrahun us level pey mujheko chahiye hotha hey
ki jo hamara rbac service ACCOUNT BANAREY UNKO NAMESPACE LEVEL THAK KA HI ACCESS chahiye
agar aek user cluster level pey bantha hey cluster role an binding use hothi hey kiunki pura cluster ka monitor , logging sab karna hotha hey
ek dir create karenge mkdir dadhboard

4. Setting Up the Kubernetes Dashboard

Metrics server

Deploy the Dashboard Apply the Kubernetes Dashboard manifest:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml

is ney ek namespace banaya service banaya , secret , role banaya , cluster role b banaya
cluster role already bangaya toh isliye humko user create karkey us role ko role bind karna hey

Create an Admin User Create a dashboard-admin-user.yml file with the following content:

vim dashboard-admin-user.yml 

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

Apply the configuration:

kubectl apply -f dashboard-admin-user.yml

ab mai jo mera cluster rehtha hey na us chalaney key liye b mehnat padtah hey
Get the Access Token Retrieve the token for the admin-user:
```
  kubectl -n kubernetes-dashboard create token admin-user
```
Copy the token for use in the Dashboard login.

Access the Dashboard Start the Dashboard using kubectl proxy:
```
  kubectl proxy #--port=8001 --address=0.0.0.0 --accept-hosts="*"
```
Open the Dashboard in your browser:
```
  http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
```
Use the token from the previous step to log in.
you have dashboard - ab cd nginx → kubectl apply -f namespace.yml
- kubectl apply -f persistentvolume.yml
  
  * kubectl apply -f apply .
  
  * kubectl get all -n nginx —> aap dashboard mey jakey namespace select karlo aap sab dikhega - ae dashboarrd sab metric server, monitoring and logging

Custom resource definition:

agar bath kare custom resource definition ki ki sabsey pehle samajna padtha hey resource kya hotha hey
toh jo k8s mey pods. services, deployment , services ae sab resources hothi hye
ae saare resources kubernates ko already pata hey ae servic e, namespace , ae kya hey is ka structure kya hey isko kaisa patha agar aapko aisey resources banana hey jo k8s mey nahi hey by default like custom resource
mai tws mey bohot saarey batches letha hun mai khudke liye mai saare batches ko store kar sakun.
jo pod ka structure aap khud key liye karna hey usliye custom resource definition use aathey hey

jaisa kind: pod hey vaisa kind:DevOpsBatch -k8s ko ae kya resource karkey pod ka jo taml structure hey vahi structure mey mai hava mey create nahi kar saktah mujhey is spec resource ko define karna padtha hey

custom resource sturuction is a definition with that we can create our own resources

cd crd: vim devops-crd.yml

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: devopsbatches.trainwithshubham.com
spec:
  group: trainwithshubham.com
  names: 
    plural: devopsbatches
    singular: devopsbatch
    kind: DevOpsBatch
    shortNames:
      - junoon
      - batches
      - tws
  scope: Namespaced
  versions:
  - name: v1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec: 
            type: object
            properties: 
              name:
                type: string
                description: "This is the name of the DevOps Batch"
              duration:
                type: string
                description: "This is the duration of the DevOps Batch"
              mode:
                type: string
                description: "This is the mode of the batch eg. Live/Recorded"
              platform:
                type: string
                description: "This is the platform of the batch"

kubectl apply f crd.yml
kubectl get crd → is ka faida aap isko use karkey ek devops custom resource bana sakthey hey

vim devops-cr1.yml

kind: DevOpsBatch
apiVersion: trainwithshubham.com/v1
metadata:
  name: junoon-batch-9
spec:
  name: DevOps-Zero To hero Junoon Batch 9
  duration: 3 months from 25th January 2025
  platform: trainwithshubham.com
  mode: Live as always

vi devops-cr2.yml

kind: DevOpsBatch
apiVersion: trainwithshubham.com/v1
metadata:
  name: junoon-batch-10
spec:
  name: DevOps-Zero To hero Junoon Batch 10
  duration: 3 months from 25th January 2025
  platform: trainwithshubham.com
  mode: Live as always

Operators: basically programming mey hothey hey like argocd , prometheuus operators , aur crd key resources ko programm kar sakthey hey

Kubernates-api : we can access resources via kubernates-api

kopf -. python framework with the help of python code you can write

image scanning: we can do by using docker scout or trivy

secret-encryption: we have seen base64 encode and decode,

we cover network policies in cicd gitops , we will see eks in ci and cd , debugging and troubleshoot we have seen , resource utilization

kubectl top pod - resource utilization

kubectl top node

Helm:

dosto ab thak itnney saari manifest files. overwhelming hojatha ithna manifest files likhna uska kind likhna , template likhna
- ek aisa tool hey jiskey vajhasey aap ae sab manifiest likhnesey muft hosakthey ,mey apt karkey package install karthey hey vaisaey hi aur har ek environmnet key liye laag sey manifest file banana nahi padegi sab kuch package rahega
- ae tool ka name hey helm
- helm aek package manager hey kubernates ka - ae package kya hey
- like ubuntu mey apt key vajhasey aap koi packages install karthey hey aisey hi helm key through cheezon ko install karlethey ho using kubernates manifest file

toh helm jo hotah hey ek package manager for k8s, toh aapko for ex: toh bahut saarey manifest files banana padtha hey for nginx , mysql , notes-app ka

us nginx key liye , aap deployment , service, ingress , hpa, configMap , sercret ae saarey cheeze har application key liye likhthey hey right
har cheez jab same hi honi hey kiun na hum ek package manager use karley aur vo package manager in sab cheezon ko install karega

Create one folder helm and helm key andar install karney wala hun helm charts - mkdir helm → cd helm

lets say nginx mey sab resources .yml hey aapney khud banaya aur aehi cheez automated sey ek pacjage manager key sath banani hothi hey toh helm sey kar sakthey hey pehle isko install karna padtha hey

This Helm chart deploys an Apache HTTP Server on a Kubernetes cluster.

Prerequisites

Kubernetes cluster running (local, cloud, or KIND).
kubectl installed and configured.
Helm 3 installed. Install Helm with:

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

helm version  # helm go mey likha hey

Chart Structure

apache/
├── Chart.yaml         # Chart metadata
├── values.yaml        # Default values for customization
└── templates/         # Kubernetes resource templates
    ├── deployment.yaml
    └── service.yaml

humlog ek helm chart banayanege
- helm create apache-helm → mai ek chart banara hun
- jiskey andar mai apache chalaunga
- ls - cd apache-helm → ls hum ko milega chart.yaml , charts , templates , values.yml
- tree
- templates hey andar - sab resources .yml banadiya thats teh beauty of helm
- manifest files likhna zaroori nahi → cd templates → vim deployment.yml - humko sirf values dalni hey bus
- aap jo b change karna hey values mey jakey change karo idhar reflect hojayega
- kuch b nahi likhna humko , helm ney banakey dega
- un templates ko use karna hey values values.yml sey jathey hey
- vim values.yml - isme jo b values hey vo udhar pey inject ho jathey hey
- humko jo b chahiye vo values.yml mey dal sakthey hey
- vim chart.yml - ae kya hey -ae jo b hum chart banare lets say apache, nginx kuch b uska info idhar rehtha hey
ab mujhey is chart ko package karna gey
helm package . → is package ko mai export , import kuch b kar saktha hun → cd ..
helm package apache-helm
ab mujhey is chart ko install karna hey -
→ helm install mai aapney chart ko kya name dena chahtha hun like lets say nginx key jagah apache dena chahtha hun
ab mujhey iska dev , prod , stae environment banana hey
mai isko name dey saktha hun → helm install dev-apache apache-helm
default name space key andar apache mey running hey
→ kubectl get deployment → kubectl get pods
helm uninstall dev-apache → saare deployment hojathey hey
jab b aap koi helm chart install akrthey ho sathi sath mey namespace b dedo
helm install dev-apache apache-helm dev-apache —create-namespace
agar aap production key banare -helm install prd-apache apache-helm prd-apache —create-namespace
mai vales.yml mey 3 replica daldunga ab mai prd upgrade karna chahtha hun
vim chart.yml - change version 1.16.1
helm package apache-helm
helm upgrade prd-apache ./apache-helm -n prd-apache
kubectl get pods -n dev-aapche
kubectl get pods -n prd-apache # so it is eady to maintain the multiple environments
ab question aatha hey mera revision number 1 jismey 3 ki jagah 2 hi hey , uspey roll back karna hey
helm rollback prd-apache 1 -n prd-apache
helm create node-js-app
cd node-js-app → vim chart.yml → vim templates/service.yml
- vim values.yml → mai kya kar saktey hey - trainwitshubham/notes-app image → tag: latest ; port:8000 targetPort: 8000 ;
- helm package node-js-app
- helm install dev-notes-js-app -n node-js-app dev-node —create-namespace
  - dev-env name , chart name - node-js-app and namespace
- kubectl get pods -n dev-node
- kubectl get svc -n dev-node
- kubectl port-forward svc/dev-node-js-app 8000:8000 -n dev-node —address=0.0.0.0
- within minutes you can run your microservices , you can instal anything and can run
- help repo - google → chalo through helm argocd dekhenge

ArgoCD: search in google argocd -select helm

humko repo add karna padtha hey isliye humko repo serahc karna hey
helm search repo nginx
helm search repo argocd
helm repo add stable https://charts.helm.sh/stable
ab aapkey system mey ae stable repo add kardethi hey

          helm install nginx-helm oci://registry-1.docker.io/bitnamicharts/nginx

- helm uninstall nginx-helm

helm uninstall prd-apache -n prd-apache
lets say you are asked to install mongodb

          helm install mongodb-helm oci://registry-1.docker.io/bitnamicharts/mongodb -n mongo --create-namespace

- kubectl get pods -n mongo # agar mai karunga - ek init container chalra hey - aapka mongo db container chalra hoga

kubectl exec -it mongodb-hel, -n mongo — bash → ls → monosh
aisey helm key through ham cheezon ko deploy kar sakthey hey
helm list -n mongo
helm uninstall mongodb-helm -n mongo

Side car containers vs Init containers - iska parent topic hey pod - generally hum pods create karthey hey , ismey containers chalathey hey usmey sey b init container hotha hey ek side car container hotha hey

diff kya hey → aisa samjo ki ek pod hey jo aap pod key liye manifest file banathey hey iska kind rehtha hey pod
bus uskey andar spec iskey andar containers rehthey hey ek main container rehtha hey jismey aapka application chalna chaiye , bakhi init and side car containers main container ko help karnekey liye rehtha hey
jaisey aap db banare , us db ko bananey key liye main container liya usmey image daldiya - is postegres ki jo main container bannesey pehley lets say koi onnections , path kuch b chahiye hoga - us cheez ko initialize karnekey liye ek init container lagtha hey
init container jo main container chalnesey pehley jo cheeze chahiye vo init cont dey detha hey
pehle folder banega init container sey phir main container run hoyega

side car : jaisey batman , batmen pehley akeley ladtha tah ab kisi ka sath liye - ek simple single manifest file banegi iska kind pod ismey spec mey 2 containers rehthey hey - kuch logs copy , yah as aa helper main containers key sath help conatiner jaisa rethate hey

vim init.yml

kind: Pod
apiVersion: v1
metadata:
  name: init-test

spec:
  initContainers:
  - name: init-container
    image: busybox:latest
    command: ["sh","-c", "echo 'Initalization started ...'; sleep 10; echo 'Initization completed.'"] 

  containers:
  - name: main-container
    image: busybox:latest
    command: ["sh","-c", "echo 'Main container started'"]

kubectl apply -f init.yml
kubectl logs init-test -c init-container -c main-container
kubectl logs init-test -c init container
kubectl get pods


kind: Pod
apiVersion: v1
metadata:
  name: sidecar-test

spec:

  volumes:
  - name: shared-logs
    emptyDir: {}

  containers:
  # produce logs
  - name: main-container
    image: busybox
    command: ["sh","-c", "while true; do echo 'Hello Dosto' >> /var/log/app.log; sleep 5; done"]
    volumeMounts:
    - name: shared-logs
      mountPath:  /var/log/

  # display logs
  - name: sidecar-container
    image: busybox
    command: ["sh","-c", "tail -f /var/log/app.log"]
    volumeMounts:
    - name: shared-logs
      mountPath:  /var/log/

side by side help karna iska kaam hey
kubectl apply -f side-car.yml
kubectl get pods
kubectl logs side-car-test -c main-container
kubectl logs side-car-test -c sidecar-container
init cont - google k8s

service mesh :

service mesh , dekho service mesh - service na ek mess hi hothi hey yaha aek pod hey iski service chalri hogi
suppose ek three tier application hey like voting , result, worker iskey alag alag services and ports like 5000, 50001 ,

aapkey pass zomato app hey - jismey ek service chalri hey menu , vaha sey dusri service jatha hun orders , aur events aisey bohot saarey services hey , delivery , hotel kaafi complex app der saari services chalri hogi ae saari services kaha sey jayegi isi cheez key aapko chahiye hotha hey ek gateway

us gateway key through saari services jayengi , pune mey hinzwadi mey bohot zyada traffic rehtha hey
ae traffic man controll karta hey kounsa traffic kidar redirect karna hey , is traffic ko redirect karnekey liye we need a tool called service mesh
service mesh ek gateway dey detha hey jissey aap services ko route , redirect kar sakthey hey basically complex service ko aap badiya ek dam distribute karsakthey ho jaisey ki har complexity simplyfy hogai
service mesh key liye ek popular tool hey istio , jo ki kaafi popular tool hey
go to ggole istio demo - lets install - clieck on instll - platform specific - > search kind - clieck on kind-istio
- cd istio → cd istio-practice

Download Istio

Go to the Istio release page to download the installation file for your OS, or download and extract the latest release automatically (Linux or macOS):
```
 $ curl -L https://istio.io/downloadIstio | sh -
```
Move to the Istio package directory. For example, if the package is istio-1.26.1:
```
 $ cd istio-1.26.1
```
- cat readme.md
- istio uses envoy which is a sidecar container /proxy which allows → vo aapkey jithney b services rehthi hey un sarey services key logs ko inject kartha hey
- istiod ek control plane service discovery
The installation directory contains:
- Sample applications in samples/
- cd samples , cd .. , cd bin → ./istioctl
- The istioctl client binary in the bin/ directory.
Add the istioctl client to your path (Linux or macOS):
```
 $ export PATH=$PWD/bin:$PATH
```
- jaisa kubectl key commands kuectl sey chalathey vaisa hi ./istioctl
- cd bin → mv istioctl /usr/local/bin
Install Istio using the demo profile, without any gateways:

$ istioctl install -f samples/bookinfo/demo-profile-no-gateways.yaml -y

Add a namespace label to instruct Istio to automatically inject Envoy sidecar proxies when you deploy your application later:

$ kubectl label namespace default istio-injection=enabled

Install the Kubernetes Gateway API CRDs

The Kubernetes Gateway API CRDs do not come installed by default on most Kubernetes clusters, so make sure they are installed before using the Gateway API.

Install the Gateway API CRDs, if they are not already present:

 $ kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || \
 { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v1.3.0" | kubectl apply -f -; }

Deploy the sample application

You have configured Istio to inject sidecar containers into any application you deploy in your default namespace.

Deploy the Bookinfo sample application:

 $ kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

The application will start. As each pod becomes ready, the Istio sidecar will be deployed along with it.

$ kubectl get services
- kubectl port-forward svc/productpage 9080:9080 
agar merey pass 50 micro services kithno ka service manage and alag karun , url manage ahi sakthey isliye isio lagtha hey 

$ kubectl get pods

Validate that the app is running inside the cluster by checking for the page title in the response:

$ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -sS productpage:9080/productpage | grep -o ".*"

Open the application to outside traffic

The Bookinfo application is deployed, but not accessible from the outside. To make it accessible, you need to create an ingress gateway, which maps a path to a route at the edge of your mesh.

Create a Kubernetes Gateway for the Bookinfo application:
```
 $ kubectl apply -f samples/bookinfo/gateway-api/bookinfo-gateway.yaml
 gateway.gateway.networking.k8s.io/bookinfo-gateway created
 httproute.gateway.networking.k8s.io/bookinfo created
```
By default, Istio creates a LoadBalancer service for a gateway. As we will access this gateway by a tunnel, we don’t need a load balancer. If you want to learn about how load balancers are configured for external IP addresses, read the ingress gateways documentation.

Change the service type to ClusterIP by annotating the gateway:

 $ kubectl annotate gateway bookinfo-gateway networking.istio.io/service-type=ClusterIP --namespace=default

To check the status of the gateway, run:

 $ kubectl get gateway
 NAME               CLASS   ADDRESS                                            PROGRAMMED   AGE
 bookinfo-gateway   istio   bookinfo-gateway-istio.default.svc.cluster.local   True         42s

Access the application

You will connect to the Bookinfo productpage service through the gateway you just provisioned. To access the gateway, you need to use the kubectl port-forward command:

8$ kubectl port-forward svc/bookinfo-gateway-istio 8080:80

View the dashboard

Istio integrates with several different telemetry applications. These can help you gain an understanding of the structure of your service mesh, display the topology of the mesh, and analyze the health of your mesh.

Use the following instructions to deploy the Kiali dashboard, along with Prometheus, Grafana, and Jaeger.

Install Kiali and the other addons and wait for them to be deployed.

 $ kubectl apply -f samples/addons
 $ kubectl rollout status deployment/kiali -n istio-system
 Waiting for deployment "kiali" rollout to finish: 0 of 1 updated replicas are available...
 deployment "kiali" successfully rolled out

Access the Kiali dashboard.
```
 $ istioctl dashboard kiali
```
for i in $(seq 1 100); do curl -s -o /dev/null "http://localhost:8080/productpage"; done

Uninstall

To delete the Bookinfo sample application and its configuration, see Bookinfo cleanup.

The Istio uninstall deletes the RBAC permissions and all resources hierarchically under the istio-system namespace. It is safe to ignore errors for non-existent resources because they may have been deleted hierarchically.
```
 $ kubectl delete -f samples/addons
 $ istioctl uninstall -y --purge
```
The istio-system namespace is not removed by default. If no longer needed, use the following command to remove it:
```
 $ kubectl delete namespace istio-system
```
The label to instruct Istio to automatically inject Envoy sidecar proxies is not removed by default. If no longer needed, use the following command to remove it:
```
 $ kubectl label namespace default istio-injection-
```
If you installed the Kubernetes Gateway API CRDs and would now like to remove them, run one of the following commands:
- If you ran any tasks that required the experimental version of the CRDs:
```
  $ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v1.3.0" | kubectl delete -f -
```
- Otherwise:
```
  $ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/cr
```

Projects:

projects jo hum resume mey add kar sake

Chat application on kubernates -3 tier application - jaha pey front end react js and backend nodejs pey chalra and mongodb pey chalra
rule key muthabik - docker container aajayega - docker container sey bantha hey pod - iski deployment → is cheez ka service

miniqube start —driver=docker

kubectl get pods

cd projects:

shubham -full stack chat app → git clone https://github.com/LondheShubham153/full-stack_chatApp.git
login to dockerhub
cd backedn → docker build -t vaseem143/chatapp-backend:latest .
image tag and push the image
same with frontend also
now by using these two images and create the pod

create a namespace.yml

  apiVersion: v1
  kind: Namespace
  metadata:
    name: chat-app

kubectl apply -f namespace.yml

kubectl get ns

vi backend-deployment.yml

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: backend-deployment
    namespace: chat-app
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: backend
    template:
      metadata:
        name: backend-pod
        namespace: chat-app
        labels: 
          app: backend
      spec:
        containers:
        - name: chatapp-backend
          image: trainwithshubham/chatapp-backend:latest
          ports:
          - containerPort: 5001
          env:
          - name: NODE_ENV
            value: production
          - name: MONGODB_URI
            value: "mongodb://mongoadmin:secret@mongodb:27017/dbname?authSource=admin"
          - name: JWT_SECRET
            valueFrom:
              secretKeyRef:
                name: chatapp-secrets
                key: jwt 
          - name: PORT
            value: "5001"

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: frontend-deployment
    namespace: chat-app
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: frontend
    template:
      metadata:
        name: frontend-pod
        namespace: chat-app
        labels: 
          app: frontend
      spec:
        containers:
        - name: chatapp-frontend
          image: trainwithshubham/chatapp-frontend
          ports:
          - containerPort: 80
          env:
          - name: NODE_ENV
            value: production

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: mongodb-deployment
    namespace: chat-app
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: mongodb
    template:
      metadata:
        name: mongodb-pod
        namespace: chat-app
        labels: 
          app: mongodb
      spec:
        containers:
        - name: chatapp-mongodb
          image: mongo:latest
          ports:
          - containerPort: 27017
          env:
          - name: MONGO_INITDB_ROOT_USERNAME
            value: mongoadmin
          - name: MONGO_INITDB_ROOT_PASSWORD
            value: secret
        volumes:
        - name: mongo-data
          persistentVolumeClaim:
            claimName: mongodb-pvc

  apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: mongodb-pv
    namespace: chat-app
  spec:
    accessModes:
      - ReadWriteOnce
    capacity:
      storage: 5Gi
    hostPath:
      path: /data

  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: mongodb-pvc
    namespace: chat-app
  spec:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi

kubectl apply -f pv.yml
kubectl get pv -n chat-app
kubectl apply -f pvc.yml
kubectl get pvc -n chat-app
kubectl apply -f mongodb-deployment.yml
kubectl get pods -n chat-app
kubectl apply -f frontend.yml
kubectl apply -f backend.yml

vi backend-service.yml

apiVersion: v1
kind: Service
metadata:
  name: backend
  namespace: chat-app
spec:
  selector:
    app: backend
  ports:
  - port: 5001
    targetPort: 5001

kubectl apply -f backedn-service
kubectl apply -f frontend -service
kubectl apply - ffrontennd-deployment.yml
kubectl get pods -n chat-app

apiVersion: v1
kind: Service
metadata:
  name: frontend
  namespace: chat-app
spec:
  selector:
    app: frontend
  ports:
  - port: 80
    targetPort: 80

Jenkins in One shot .

Vaseem Shaik — Sat, 31 May 2025 18:43:00 GMT

Introduction to Jenkins & CI CD
Jenkins Setup On VM (AWS EC2)
Jenkins Installation
Jenkins UI/Dashboard / Jobs
Jenkins Freestyle Project
📍 Declarative Pipeline
Jenkins Agents (Multi-Node)
Declarative pipeline Demo
Shared Libraries
User Management In Jenkins (Role Based)
CI/CD Project with Kubernetes, ArgoCD, Prometheus
Shared Library Realtime Usecase + Demo
DevSecOps with OWASP, Trivy, Sonarqube
Email Notification on Pipeline Fail/Pass

Introduction to Jenkins & CI CD

Jenkins hotha kya hey , jenkins ko samajney key pehley aapko samajna padtha hey CI & CD kya hotah hey

Dekho kaisa rehtha hey har ek org mey project hey - github aek tool hey project ko store kar sakthey hey scm key liye use hotha hey , ae code developer likhey dal detha hey github mey - ae docker pey build hotha hey aur k8s or ec2 mey deploy hotha hey - ek baar deploy hogaya usko monitor karthey hey prometheus sey

jab b developer code push karega phir sey build , test , deploy and monitor hoyega ae cheez continusly honi chahiye

aek ek badh ek baar baar honi chahiye , lets sey aapka build fail hoayega toh aagey deploy nahi hoga monitoring mey b dikkat nahi aayegi
aapka build pass hua toh turanth aapko production thak paunchana hey - time to markey kamm karna hey - isko kamm kartha hey devops engineer—> is puri process ko kam sey kam time mey paunchana
ae jo code , build , deploy , monitor is cheez ko automate kar skathey hey using jenkins
jenkins ka kaam hey in saare cheezon ko automate karna hey
agar devops ka logo dekhey aap toh code , build , test and deploy - as in infinity symbol - infinity process
Jenkins aek aisa tool hey jo aapko CI & CD detha hey -→ continuos integration , and continuos deployment and delivery .
- CI : jaha pey test cases run hothey hey , checkout code , building the code , running the test cases .
- CD: continuos delivery - amazon pey kuch order karthey ho vo delivery hothi hey manually aap order karthey hey right , that time you use continuos delivery .
- kabhi kabhi instagram mey auto update hojatha hey - vo basically deployment - without manual intervension vo continuos deployment.

Jenkins setup on Vm :

Jenkins is a GUI tool hey -→ go to aws -→ create and jenkins-master →select o.s and create a machine & connect to the machine .
Go to google —> install jenkins - select linux —>
You wil get commands to run , copy and run the commands.
ae ek gui based tool , devloped by java - jo aapko ci and cd ko automate kar saktha hey
jenkins install karnesey pehley java install karna padega
sabsey pehle -→ sudo apt-get update -y
sudo apt-get install openjdk-17-jdk -y
to ensure java is there or not -→ java -version.
Now execute the commads to install jenkins from long-support.
sudo apt-get install jenkins
aapey system mey status dekhna jenkins ka -→ sudo systemctl status jenkins .
hum chahthey ho agar mera system restart ho jenkins b restart ho
#systemctl enable jenkins
- ab mera jenkins chalra hoga port number 8080 -→ enable the 8080 port number
aapka jenkins aapkey samney hey - unlock jenkins - install suggested plugins
give username and password : admin and admin -→ ab ek admin user banya
Now jenkins is ready
Chalo jab 2018 mey mai pehlri baar dekha hey shock hoayatha - ae sab dashboard kya hey karkey
This is jenkins ka ui , ae ui browser pey chaltha hey
iskey andar aap - job banasakthey ho , agents , cloud configure kar sakthey ho
Job: jenkins mey kuch b automate karana hey ci & cd pipeline banani hey vo ek job , git configure vo ek job , scripting vo ek job , jenkins ka logo dekho vo ek buttler - vo wait karra hum ko order do hum vo kaam karthey hey

jenkins mey hum log generally jobs hum log create karthey hey

Create a job - item types - ab is item ka name → first job
- is job ka name dediye - is item ka types - ispey automate kar sakthey hey
- pipeline mey declarative pipelines banana padtha hey
- multi-configuration - aapko multiple environmnets mey like dev , test , prod mey karna hey
- multi -branch - git mey bohot saarey branches rehthey hey us branch mey automation kara na hey
- organization mey ek folder bana diya - us org folder key andar aap bohot saare job sbana saktehy hey
ab hum free style click karney key badh - ek form khultha hey - aapko kya kaam kara na hey ismey configure kara na hey
- - job key builds rehthey hey - merey builds ko discards karna hey - like storage ko bacha sakthey hey
    
    * log rotation aisa hey aapley pass 10 days key builds hey usme aapko sirf 2 days key store karna hey aap log rotation use kar sakthey hey
    
    * github- project -→ idhar jo project karna hey git ka project url idhar dey sakthey hey
    
    * parameterized- koi parameters pass karreho
    
    * throtle builds: agar baar baar click karenge phir b ek build pe ruk jayega
    
    * concurrent build - agar aapko parallel mey chalana hey
- scm - github sey idhar project use kar sakthey hey
- build triggers:
  - is builds ko chalaney key liye scripts sey chalana hey builds remotely use karthey hey
  - ek job ko khatam honekey badh chalana chathey ho isko chala sathey jey - build after other jobs
  - build periodically - daily 6 baje aapka build chalna chathey ho
  - github hook - aap chahthey ho developer push kiya pipeline build chal jaye is option use kar sakthey hey
- Build environment:
  - jismey secret use karna hey - idhar use kar sakthey hey
- build scripts:
  - execute scripts :
    - aapko ek command shell open hoyega - ismey jo b command dalege chal jayega
    - jo b idhar diya - vo chalayega .
- go to dashboard and click on build - continuos delivery manually click karrey

Declarative pipelines:

aisa samajlo ae aapka tap hey - ae aapka garden hey ab is garden mey pani jana hey - aap pipe laga dogey - is garden mey pani jana hey aap sprinklen laga dogey
- aapko na ae pani shud hokey jana hey filter laga dogey
- aek source sey ek destination mey jarey ho using piepeline - beechmey stages lagarey
- jenkins b ci and cd pipelines banatha hey - stages key through pipeline bana key destination bana sakthey ho
- go to dashboard click on new item - name demo-cicd —> selct pipeline and click on ok and save.
click on configure - pipeline script - try sample pipeline first → hello world - heloo name ki stage hey
Aap key pass multi -stage pipeline b ho sakthi hey - multiple stages
- output aapko stage by stage perform hotha hey
- isko bolthey hey groovy syntax - isko build kardo
- cd /var/lib/jenkins/workspace mey aapka job store kiye rehthey

Agents :

idhar hum agent
any rakhey - industry mey master and nodes rehtha - jenkins master rehta hey - ae workload nodes mey distribute kartha hey

ae aapka jenkins ka master server rehtah hey - jenkins nodes ko kaam deyega aur yaah pey actually job run hoaga , jisseki jenkins mey memory issues na aaye jenkins safe rahey
build in node mastlab jaa pey jenkins hey vahi pey builds job chalrehongey usko bolthey built in nod e- it is not safety and industry standards so distribute the laod
go to aws create the three VM instance , mai idhar nodes agnets banara hun agent kya hotha hey → aap generally build in nodes m=pey jobs nahi chalathey - aap order dethey nodes ko , agents pey jobs chalthey hey
aap key agents mey java hona chahiye → connect to vm install the java
→ ae agent connect honge kaisey → toh jenkins ssh through connect karega -aap ek server sey dusre server ko connect kar sakthey hey throuh ssh -using public na private key
private stores itself and pulich should be there in agent
go to the master machine → cd ~/.ssh → ssh-keygen → ls → cat id → copy it and go to the agent → under the → cd ~/.ssh
go to the jenkins - dashbode - agents - click on new node → agent-vaseem → permanent agent - create it → no of executers - ek sath mey kithne jobs ek time pe chalan achathey hey - 1
- remote dir: /home/ubuntu
- labels: label jo hotah hey isi sey saara kaam hotah hey pipeline mey ae label name agent any key replace akrney label name dalthey
- use this node as much as possible - launch via ssh → host ip → copy ip address → credentials agent ki jo connect karnekey liye key - mai aapney master mey aunga - private key copy karlunag - add credentials - select ssh uername with privatekey → ubuntu-key → description → username: ubuntu → enter directly - paste it
  
  -→ Host-key stategy kardo non host
- host -ip - click on agent - failed to connect →ssh authentication failed because agent mey public key nahi hai toh public key agent .ssh mey authorized keys mey add kardo .
- ab agent mey jakey click on launcg agent - now it will successfully connected .
ab pipeline mey jakey agent any key place mey { label: “vinod”}
vinod us agent ka label hey - usmey vahapey koi b folder nahi hey → agar mai pipeline run karunga toh udhar pey folder banna chahiye thab mai confirm kar saktha hun mera ci and cd process achesey chalra .

Declarative pipeline:

click on new item → djanjoCICD

Description - this is ci and cd pipeline
github ki url dedunag → https://github.com/LondheShubham153/django-notes-app.git
DisplayName: Django Notes App
hothub hook trigger for GITScm polling click
Advanced description , pipeline - ab pipeline likhna sikhanege

@Library('Shared')_
pipeline{
    agent { label 'dev-server'}

    stages{
        stage("Code clone"){
            steps{
                sh "whoami"
            git url: "https://github.com/LondheShubham153/django-notes-app.git",branch:"main"
            echo " This is cloning the code"
            }
        }
        stage("Code Build"){
            steps{
            echo " Building the code"
            sh "whoami"
            sh "docker build -t notes-app:latest ."
            }
        }

        stage("Push to Docker Hub"){
            steps{
                withCredentials([usernamePassword(credentialsId:"dockerHubCred",passwordVariable:"dockerHubPass",usernameVariable:"dockerHubUser")]){
                sh "docker tag notes-app:latest ${env.dockerHubUser}/notes-app:latest"
                sh "docker login -u ${env.dockerHubUser} -p ${env.dockerHubPass}"
                sh "docker push ${env.dockerHubUser}/notes-app:latest"
                }
            }
        stage("Deploy"){
            steps{
                echo "Deploying the code"
                sh "docker run -d -p 8000:8000 notes-app:latest"
            }
        }
            stage("Deploy"){
            steps{
                echo "Deploying the code"
                sh "docker compose up -d "
            }
        }

    }
}

pehle pipeline → phir ae pipeline aek agent mey chalna hey agent aur label likhdo - phir hamari pepeline stages pey chalengi toh stages likhdo
ab build kardo → mujhko aisa chahiye ki idhar pipeline dikhana hey iskey liye pluglins download karo
pipeline stage view - plugin usko kardo install
ab iskey andar actual masala dalna hey - code vali stage mey github ka code clone hojaye
ab build iskey liye docker chahiye - isko mai agent mey docker install kardunga → sudo usermod -aG docker ubuntu && newgroup docker -→ docker ps
ab merey pass hey dockerfile docker file sey docker image banthi hey
echo “whoami” dekey user dekhlo → ubuntu ko permission dedo -→ agent mey sudo reboot karenge
mater mey jenkins ko restrat kardo -→ sudo systemctl restart jenkins - > ab build hogi → ab run karengey → configure mey jakey
deploy ka configure karkey - build now kardo - now take your instance ip : and enable the port 8000 - now browse karo -→ ae application jenkins pipeline key sath deploy kiya - aur badiya kaisey kar sakthey hey
docker compose key through b deploy kar sakthey hey → docker compose - agent mey install kardo
docker run hum production mey run nahi karthey - docker compose up -d agar container hey toh kuch nahi kahega , agar container nahi hey create karega
aur kya acha kar sakthey hey - iskoa jo build image hey -isko dockerhub mey push kara sakthey hey -→ iskey liye docker login karna padtah hey iskey liye credentials chahiye rehtah hey
mujey deploy karnesey pehley aur ek stage aajayegi push to docker hub
docker image tag default image sey docker user name sey tag karey aur → docker login kare - merko credentials dalna hey isko mai expose nahi kartha toh variables banakey pipeline mey reffer karunga -→ credentials binding karna padtha hey - liek hide karkey kartha hun env variables mey
isko save karlo -→ dashboard mey jav → credentials mey javo - global credentials mey - username and password - docker mey ek token banalo personal access token → give full access
idhar credentials mey dalko → id aakey dockerHubCred→ description mey b → dockerHubCred ae id hey hey ismey andar mera username and password mey hey → ae cheez mai mrery pipeline mey use karunga
push to docker hub mey .
- ab mai mere docker hub mey images hey - abhi compose file build barey baar hora instead of that mera image lena
- aur githubwebhook b dekhenge - aap chahthey ho github mey change hua automatic lena hey - vo karangey webhook - ek webhook trigger hojaye - github jenkins url ko hit karega agar kuch update hua toh thab jenkins lekey automatically run karega
- jenkins ka url lelo jo b jenkins chalra hey →go to settings → repo settings - webhook → add webhook → payload url mey jenkins url /github-webhook/application/x select→ → no ssl - send me everything → add webhook
- mera webhook configure kiye -jenkins mey githubwebhook hook trigger enable rakhna hey
- mai docker compose mey image rakhunga instead of build. → this is called continous deployment.

Shared Libraries:

aapne jo likha na stages, code , build , push to dockerhub → mujheko kalko aur piepline likhana hogi mujheko puri mehnat vapas likhunga
- mujhey kal ko kuch deploy karna ae mehnat phir sey likhunga
- shared library kya hotha hey jenkins mey - ek repo rehthi hey - jiskey andar groovy ka code hotha hey us code ko aap reuse kar sakthey hey apni pipeline mey
- Repo → groovy code → reuse(in pipeline)
- ae jo pipeline iwth credentials aapko ae reuse karna hey - shared library create kardo
- ek new repo create kardo - name rakho - jenkins-shared-libraries → ek small snippets reuse kar sakthey hey
- usmey ek file add karni hye - ab shared libraries hothi hey vars - vars name ka folder banado
- mai chahtha hun shared library file like heloo,groovy
```
  def call(){
      echo " HELOO DOSTO"
  }

  clone:

  def callString url , String branch {

         git url: "${url} , branch:"${branch}"
              echo " This is cloning the code"
              }

  def call(String ProjectName , String ImageTag , String DockerhubUser){

              sh " docker build -t ${DockerHubUser/${ProjectName}:${ImageTag} ."    

               {

  def call(String Project, String ImageTag, String dockerhubuser){
    withCredentials([usernamePassword(credentialsId: 'dockerhubCred', passwordVariable: 'dockerHubPass', usernameVariable: 'dockerHubUser')]) {
        sh "docker login -u ${dockerHubUser} -p ${dockerHubPass}"
    }
    sh "docker push ${dockerHubUser}/${Project}:${ImageTag}"
  }
```
  - iskey andar mey ek function banaunga def jisko call karunga - parameter open parameter close
  - ab ae github sey merey jenkins mey kaisa paunchega - ab jenkins - manage jenkins - system — system wide reuse karna hey - niche - shared libraries - global trusted pipeline libraries - add library → name = Shared → default versin -branch la name aajayega → main
  - ab retrival method→ modern scm - git → repo url → save kardo
  - shared library ki repo hey jenkins mey add kiya - aap pipelines mey add nahi kiya
  - pipeline mey upar @Library(“Shared”) _ → mai ek stage(“Helo”) rakhunda mai heloo print nahi karunag call karunga → script key andar ->hello()- ae pipeline logs mey print dekh sakthey hey - ae akaha sey lera shared libraries.
  - vars mey chalo - add file → clone.groovy
    - ab pipeline mey clone stage mey
    - clone(“git url”, “main”)
    - docker-build ka groovy -→ add file →docker_build,groovy

        @Library('Shared')_
        pipeline{
            agent { label 'dev-server'}

            stages{
                stage("Code clone"){
                    steps{
                        sh "whoami"
                    git url: "https://github.com/LondheShubham153/django-notes-app.git",branch:"main"
                    echo " This is cloning the code"
                    }
                }
                stage("Code Build"){
                    steps{
                    echo " Building the code"
                    sh "whoami"
                    docker_build("notes-app","lates","trainwithshubham")
                    }
                }

                stage("Push to Docker Hub"){
                    steps{
                        withCredentials([usernamePassword(credentialsId:"dockerHubCred",passwordVariable:"dockerHubPass",usernameVariable:"dockerHubUser")]){
                        sh "docker tag notes-app:latest ${env.dockerHubUser}/notes-app:latest"
                        sh "docker login -u ${env.dockerHubUser} -p ${env.dockerHubPass}"
                        sh "docker push ${env.dockerHubUser}/notes-app:latest"
                        }
                    }
                stage("Deploy"){
                    steps{
                        echo "Deploying the code"
                        sh "docker run -d -p 8000:8000 notes-app:latest"
                    }
                }
                    stage("Deploy"){
                    steps{
                        echo "Deploying the code"
                        sh "docker compose down && docker compose up -d "
                    }
                }

aap barey baar jenkins mey change karneki jaruri ahi - aap Jenkinsfile likhdo → and jenkins mey pipeline form scm - select - give url and Jenkinsfile - just edit and change

User management in jenkins (Role based)
- Production level mey na bohot saare users rehthey unko manage karna hotha hey .
  
  * * dashbode - manage jenkins → user ko add karna hey kaisa - user pery chalo - create karo - vaseem passwd: vaseem → DevOps wale - > email
  
  * logout karkey login karkey - ae naya recruit hey - usko mainey sign in karadiya → bhai kal ka aaya hua ladka saara acess hey , usko jenkins manage karneka b access hey - ae toh gadbad hey na
  
  * role based authorixation chahiya - agar vaseem key pass admin permission hey thab hi ae saare cheeze kar saktha hey - agar read and write hey vahi kar saktha hey jo hey
  
  * jenkins → plugins → available - roll based - install it
  
  * * manage jenkins - security - role based - authorizations - logged in users can do anything - > select role based strategy
  
  * new tool miljayega - manage and assign roles - roles to add - > viewer → job - read , discover , view: read , metrics: view -→ save the role → assign role - make vaseem as viewer or authenticated_users
  
  * mera user role based access control hora hey - overall read kardo and refresh

CI/CD project three tiwe with k8s , argocd , prometheus ,devsecops: sonarqube , trivy , owasp and email notification on pipeline pass/ fail.

ek developer github mey puch kartah hey jenins mey pull kartha hey - us code ka ci and cd - ae devsecops key practices hotah hey jaha pey - dependency check kiy ajatha hey , code quality gate ko check kiay ajatha hey , image ko scan ho hotah hey aur docker hub pey =push kareneg - ae baar image push hogayi - phor vaha sey jenkins khud ek job trigger karega jo ki cd rahega continuos delivery vo cheez jabbi aapki code change hoga uska image ko change karkey github pey dalega , aur gitops sey argocd us code ko pull karega aur ek eks cluster pey deploy karaga , ek baar ae app deploy hojayi - us key badh us app ka monitoring logging sab cheezey hogi , agar aap ka app fail hojai email jayeg a sucess hua toh b email jayega
is application ka argocd - eks - monitoring - through gashboards we can check

Ae sab cheeze karni hey

go to github - wanderlast - sabsey pehle toh samajthey hey application kya ey - ae three tier application hey - ae frontend tier - react hey , backend hey nodejs- data base hey mongodb ka hey - database mey transactiona karange input output - isliye cacheing rehthi hey - > agar aap server ko baar baar ek hi cheezonko aek cache mey store kardo - reddi sis the key value pair storage service that allows you to get document data whatever you want based on a key .
next hamey ek machine lagegi jissey sachi cheeze drive karegey - is master machine ec2 hogs jo ki kuch security groups , iams , security groups key sath configured hoga - kaisa banayenge - ek tarika hey terraform ec2.tf - ae jo terrafrom ka instance banega or ispey jenkins chalana hey node chahiye toh node baankey run karo yah - isi pey run karo → t2.medium - > ismey trivy , docke r, jenkins , sonar , owasp sab rahega
ek baar master hogaya jenkins ko configure karna hey devsecops key liye -ismey trivy , docke r, jenkins , sonar , owasp sab rahega
phir hum log set-up karenge eks -cluster →is cluster ko particular region sertup karenge
isi cluster mey hamey argocd setup karna hey jo argocd isi cluster ko update karega - jo b yaml files kubernates per deploy hojaye isliye argocd use karthey hey

Argocd: argo cd ek continuos delivery tool hey specially k8s key liye design kiya gaya hey , jaha pey kubernates key services jo b hey sync and deploy kar saktheyey - jihney apps chahiye uthna apps bana sakthey ho - app settings mey jakey aapkey repo sey connect kar sakthey hey directly manifest file aayengi aur deployment karegi

monitoring - yaha hey hum helm use karenge - kithne logon ko patha hey helm kya hotha hey - aapkey jo kubernates manifest file hothi hey - uskey package ko manage kartah ehy

us package manager through hum monitor and grafana prometeus dono single click install single click deploy karne valae hey

clone the project in the vm - with the help of terraform create a master vm

aws - iam - create one user aur - terraform - and i will make it admin acess and i will create access and secret keys and i will configure aws configure

create ssh-keygen → tera-key → teraform plan

ec2.tf

  resource "aws_key_pair" "deployer" {
    key_name   = "terra-automate-key"
    public_key = file("/Users/shubham/Documents/work/TrainWithShubham/terra-practice/terra-key.pub")
  }

  resource "aws_default_vpc" "default" {

  }

  resource "aws_security_group" "allow_user_to_connect" {
    name        = "allow TLS"
    description = "Allow user to connect"
    vpc_id      = aws_default_vpc.default.id
    ingress {
      description = "port 22 allow"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }

    egress {
      description = " allow all outgoing traffic "
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      cidr_blocks = ["0.0.0.0/0"]
    }

    ingress {
      description = "port 80 allow"
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }

    ingress {
      description = "port 443 allow"
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }

    tags = {
      Name = "mysecurity"
    }
  }

  resource "aws_instance" "testinstance" {
    ami             = var.ami_id
    instance_type   = var.instance_type
    key_name        = aws_key_pair.deployer.key_name
    security_groups = [aws_security_group.allow_user_to_connect.name]
    tags = {
      Name = "Automate"
    }
    root_block_device {
      volume_size = 30 
      volume_type = "gp3"
    }
  }

terraform.tf

    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "5.65.0"
        }
      }
    }

    provider "aws" {
      region = var.aws_region
    }

variable.tf

    variable "aws_region" {
      description = "AWS region where resources will be provisioned"
      default     = "us-east-2"
    }

    variable "ami_id" {
      description = "AMI ID for the EC2 instance"
      default     = "ami-085f9c64a9b75eed5"
    }

    variable "instance_type" {
      description = "Instance type for the EC2 instance"
      default     = "t2.large"
    }

vim ec2.tf - change the keyfile location - iam making a master machine

You can do from terraform or also manually
terraform init
terraform plan → once its created - connect to the instance.
ae master machine mey kya kya cheeze karna hey

sudo apt-get update -y
- One by one i should setup now
- sudo apt-get install docker.io docker-compose up -d
- sudo chmod 777 /var/run/docker.sock - alternate - sudo usermod -aG docker ubuntu && newgrp docker - refresh the group

Wanderlust Mega Project End to End Implementation

In this demo, we will see how to deploy an end to end three tier MERN stack application on EKS cluster.

Project Deployment Flow:

Tech stack used in this project:

GitHub (Code)
Docker (Containerization)
Jenkins (CI)
OWASP (Dependency check)
SonarQube (Quality)
Trivy (Filesystem Scan)
ArgoCD (CD)
Redis (Caching)
AWS EKS (Kubernetes)
Helm (Monitoring using grafana and prometheus)

How pipeline will look after deployment:

CI pipeline to build and push
CD pipeline to update application version
ArgoCD application for deployment on EKS

Important

Below table helps you to navigate to the particular tool installation section fast.

Tech stack	Installation
Jenkins Master	Install and configure Jenkins
eksctl	Install eksctl
Argocd	Install and configure ArgoCD
Jenkins-Worker Setup	Install and configure Jenkins Worker Node
OWASP setup	Install and configure OWASP
SonarQube	Install and configure SonarQube
Email Notification Setup	Email notification setup
Monitoring	Prometheus and grafana setup using helm charts
Clean Up	Clean up

Pre-requisites to implement this project:

Note

This project will be implemented on North California region (us-west-1).

Create 1 Master machine on AWS with 2CPU, 8GB of RAM (t2.large) and 29 GB of storage and install Docker on it.
Open the below ports in security group of master machine and also attach same security group to Jenkins worker node (We will create worker node shortly)

Note

We are creating this master machine because we will configure Jenkins master, eksctl, EKS cluster creation from here.

Install & Configure Docker by using below command, "NewGrp docker" will refresh the group config hence no need to restart the EC2 machine.

sudo apt-get update

sudo apt-get install docker.io -y
sudo usermod -aG docker ubuntu && newgrp docker

Install and configure Jenkins (Master machine)

sudo apt update -y
sudo apt install fontconfig openjdk-17-jre -y

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt-get update -y
sudo apt-get install jenkins -y

Now, access Jenkins Master on the browser on port 8080 and configure it.
i will setup now jenkins - install suggested plugins - > create a user - admin → passwd → vaseem akram > email address
apney ko master machine sey cluster banani hey eks bananey ley liye iam user with access and secret key chahiye
ab secret key and access key ab mai yaha pey keys laganey key liya aws cli chahiye
Create EKS Cluster on AWS (Master machine)
- IAM user with access keys and secret access keys
- AWSCLI should be configured (Setup AWSCLI)

    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    sudo apt install unzip
    unzip awscliv2.zip
    sudo ./aws/install
    aws configure

aws configure a karkey access key and secret key dedo
aws s3 ls
abhi mai eks ctl ko install karna start kardunga ekstctl key liye mujhey kubectl chahiye , kiunki eksctl ko manage kartha hey director kartha hey kubectl
Install kubectl (Master machine)(Setup kubectl )

    curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
    chmod +x ./kubectl
    sudo mv ./kubectl /usr/local/bin
    kubectl version --short --client

kubectl got installed , simillarly humko eksctl b install kardenege
Install eksctl (Master machine) (Setup eksctl)

    curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
    sudo mv /tmp/eksctl /usr/local/bin
    eksctl version

ab mai eks banaunga apney master machine mey
ussey pehley cloud formation ka us name sey kuch hey kya karkey check karni hey

Create EKS Cluster (Master machine)

    eksctl create cluster --name=wanderlust \
                        --region=us-east-2 \
                        --version=1.30 \
                        --without-nodegroup

ae 20 min time liyega jab thak jenkins mey configure mey karenge

Setting up jenkins worker node

Create a new EC2 instance (Jenkins Worker) with 2CPU, 8GB of RAM (t2.large) and 29 GB of storage and install java on it

    sudo apt update -y
    sudo apt install fontconfig openjdk-17-jre -y

Create an IAM role with administrator access attach it to the jenkins worker node Select Jenkins worker node EC2 instance --> Actions --> Security --> Modify IAM role
Configure AWSCLI (Setup AWSCLI)

    sudo su

    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    sudo apt install unzip
    unzip awscliv2.zip
    sudo ./aws/install
    aws configure

generate ssh keys (Master machine) to setup jenkins master-slave

ssh-keygen

Now move to directory where your ssh keys are generated and copy the content of public key and paste to authorized_keys file of the Jenkins worker node.

Now, go to the jenkins master and navigate to Manage jenkins --> Nodes, and click on Add node
- name: Node
- type: permanent agent
- Number of executors: 2
- Remote root directory
- Labels: Node
- Usage: Only build jobs with label expressions matching this node
- Launch method: Via ssh
- Host:
- Credentials: Add --> Kind: ssh username with private key --> ID: Worker --> Description: Worker --> Username: root --> Private key: Enter directly --> Add Private key
- Host Key Verification Strategy: Non verifying Verification Strategy
- Availability: Keep this agent online as much as possible
And your jenkins worker node is added
Install docker (Jenkins Worker)

sudo apt install docker.io -y
sudo usermod -aG docker ubuntu && newgrp docker

Install and configure SonarQube (Master machine)
hum sonarqube install karenge

docker run -itd --name SonarQube-Server -p 9000:9000 sonarqube:lts-community

trivy install karlenge abb - toh issey aapki file system scan karlenge
user , password dedeo - default admin , admin hoga
abhi cluster create nahi hua jab thak email ka configure karenge - jenkins mey

Install Trivy (Jenkins Worker)

sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update -y
sudo apt-get install trivy -y

Steps to add email notification

→ jaisey gmail open karliya - usmey manage google account - go to security - and make sure 2 step verification is on or not - make it on
- and search app - > app password - agar aapko gmail as a an application chalana hey → mujhey yaha pey ek app banana hey → jenkins - create - you will get one password - store it somewhere → now go to jenkins → manage jenkins → credentials - > gmail ka app password tha idhar dena iskey vajhasey hum gmail bhej sakthey hey → username : aap ka email , password: app-password - desrcip: gmail-password
- system → email -notification - extended email notification - idhar smt ka email likhna -=> smtp.gmail.com -→ port generally 25 agar app per chalana hey 465 use akrna —> add password -→ select ssl

aur ab → last mey email notofication test - smtp.gmail.com → default user - advance - use authenticate → dqvaseem@gmail.com , password - and 465 → test configuration →kisi aur ka email dedo
ab clusters chalu hogey toh nodes add kardenge

Go to your Jenkins Master EC2 instance and allow 465 port number for SMTPS
Now, we need to generate an application password from our gmail account to authenticate with jenkins
- Open gmail and go to Manage your Google Account --> Security

Important

Make sure 2 step verification must be on

Search for App password and create a app password for jenkins
Once, app password is create and go back to jenkins Manage Jenkins --> Credentials to add username and password for email notification
Go back to Manage Jenkins --> System and search for Extended E-mail Notification
Scroll down and search for E-mail Notification and setup email notification

Important

Enter your gmail password which we copied recently in password field E-mail Notification --> Advance

Nodes Creation:

apney ko node groups add karni hey - ussey pehley oidc rehtha hey → oepn id connect - jissey ki aapka aws account iss cluster seh bath kar paye.

Associate IAM OIDC Provider (Master machine)

eksctl utils associate-iam-oidc-provider \
  --region us-east-2 \
  --cluster wanderlust \
  --approve

Create Nodegroup (Master machine)
- aapko yaha pey - aapko ek ssh public key rehni hey eks-nodegroup-key → key pair mey jakey create kardo eks-nodegroup-key

eksctl create nodegroup --cluster=wanderlust \
                     --region=us-east-2 \
                     --name=wanderlust \
                     --node-type=t2.large \
                     --nodes=2 \
                     --nodes-min=2 \
                     --nodes-max=2 \
                     --node-volume-size=29 \
                     --ssh-access \
                     --ssh-public-key=eks-nodegroup-key

Note

Make sure the ssh-public-key "eks-nodegroup-key is available in your aws account"

- phir sey 20 min lega , jab thak owasp dekehnge

Steps to implement the project:

owasp implement karney key liye owasp ka plugin chahiye - go to dadhboard -→ plugins -→ available Plugins .
4 plugins install kana hey - blue ocean → install → restart jenkins
ab cluster bangaya - node bangaya - > now if i give kubectl get nodes - ab humney cluster bana diye

Go to Jenkins Master and click on Manage Jenkins --> Plugins --> Available plugins install the below plugins:
- OWASP
- SonarQube Scanner
- Docker
- Pipeline: Stage View
Configure OWASP, move to Manage Jenkins --> Plugins --> Available plugins (Jenkins Worker)
After OWASP plugin is installed, Now move to Manage jenkins --> Tools (Jenkins Worker)

- dependency check dhundo - > add dependency - name → OWASP - > insall automatically kardo - install from github.com → version - 10.0.3 → owasp kuch files download karega jisko 20 min lag sakthey hey , jab aap pipeline chalavoge thab

ab save karlo

ab mujhey jenkins ko aur sonar ko integrate karana hey

- toh sonar idhar hey - mujhey isko jenkins ko intgerate karana hey , humney jenkins ko github ko kaisa intgerate kiske basis pey intgerate kara ya tha - through tokens

Login to SonarQube server and create the credentials for jenkins to integrate with SonarQube
- Navigate to Administration --> Security --> Users --> Token
Now, go to Manage Jenkins --> credentials and add Sonarqube credentials:
ab sonar key token sey jenkins ko integrate kar denge - go to tools - sonar scannner - add sonar scanner
Go to Manage Jenkins --> Tools and search for SonarQube Scanner installations:

Go to Manage Jenkins --> credentials and add Github credentials to push updated code from the pipeline:

Note

While adding github credentials add Personal Access Token in the password field.

ab mujhey sonarqube ko integrate karana hey -

Go to Manage Jenkins --> System and search for SonarQube installations:
jab jenkins sonarqube pey hit karega usko ek webhook chahiye hit karekey liye
agar mujhey jenkins sey sonar ko kuch bhejna hey toh mujhey wehbook chahiye hotha hey

Login to SonarQube server, go to Administration --> Webhook and click on create

argo cd :

Ab hum log argocd ko setup karlenge - go to the repository - installations
Argocd ek cd tool - > ho kubernates specific deployments key liye kaam kartha hey
mai kaha install karunga - mai eks cluster pey karunga uska jo b kaam hey master mey rahega
- Install and Configure ArgoCD (Master Machine
- Create argocd namespace

    kubectl create namespace argocd

Apply argocd manifest

    kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Make sure all pods are running in argocd namespace

    watch kubectl get pods -n argocd

Install argocd CLI
ae jo mera argo cd hey - ae toh ui application agar same cheez cli po karni hey cli install karlethey

    sudo curl --silent --location -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/v2.4.7/argocd-linux-amd64

Provide executable permission

    sudo chmod +x /usr/local/bin/argocd

Check argocd services

    kubectl get svc -n argocd

Change argocd server's service from ClusterIP to NodePort

    kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

Confirm service is patched or not

    kubectl get svc -n argocd

Check the port where ArgoCD server is running and expose it on security groups of a worker node
Access it on browser, click on advance and proceed with

agar Fetch the initial password of argocd server

    kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

Username: admin
Now, go to User Info and update your argocd password
sign in karlo abhi → go to user info - update password .
abhi apney ko argocd ko b setup karna hey → settings - repositories
argocd aek delivery tool hey github hey sey kubernates manifest file lekey aatha hey - aur cluster pey update kar detha hey
toh mujhey ek repository chahiye hogi aur cluster chahiye hoga - > connect repo → https - > project name -default -→ clone url give here -→ conect
aisa samjo dockerhub credentails do jagah pey docker push karma hey - is valey stage pey mere wala account pey karna next stag emey dusri vala stage pey karna hey -- > toh mai shared libraries use karlunga as a function vo function mai call karletha hun
same code ko multiple pipelines mey share kar sakthey ho

Now again, Go to Manage Jenkins --> System and search for Global Trusted Pipeline Libraries:

https://github.com/DevMadhup/Jenkins_SharedLib.git

agar ae mai kisi aur node pey chalana hey toh mai label diunga

environment - sonar_home : sonar kiun diye → kiunki agar aap apney manage jenkins - tools pey javige - sonarqube tool ko sonar name sey diye hey

hamara jo project hey - ismey ae frontend - ae backend hey

toh backend ki image and front end ki image change huegi kabhi na kabhi - isi liye ae humlog parameters liyenge -

lets say version 4.7 chahiye ,mai yaha pey tags dal saktha hun toh vo vala build hojayega parameters through

usko imagetag

phir trivy , owasp , sonar analysis , sonar quality gate everything taken fron shared library

Ecporting env variables : aisa samjo ki ae jo project hey - backend - mai aapko docker compose through samjatha hun

is project mey kithni services hey - 4 services - > mongodb : container namee → image ka name: mongo → volume

aur ek - backend container - name → image: build./backend → env_file: is ka env file hey backedn file mey env file hey usmey ip port number is key environmnrt specific address hey kuch variables hey

vo file mey → ip alag hey mera ip alag hey isliye ek directory hey automations

push to docker hub shared libraries - docker push karneey liye we need credentials - docker hub pey javo - go to my account settings - aur yaha pey personal access token - generate -new-token - read write access deno generate kardo - paste the token in jenkins - managejenkins - credentials

Argo cd ka kya kaam hey - github sey manifest lekey cluster mey deploy kartha hey

how to ensure ki github pey hamari our manifest files updated hey .

deployment depends on the manifest file , if we do the continuos delivery cd - of manifest files and argocd us manifest files ko lekey deploy kardega

jo b docker image version deyegey hum us image tag sey ci ney docker image ko scan build , sonarqube aur push karega docker hub , ab cd pipeline u docker image ko kubernates manifest file mey update karega

iskey badh argocd ney us yaml files ko deploy kar dega eks-cluster mey

argocd ka kaam simple hey - manifest lena aur deploy karna

GitOps - -→ Gitops is a practice in which the code is basically pulled from github repo through that repo there are tools like argocd directly rely on git code not any pipeline .

@Library('Shared') _ pipeline { agent {label 'Node'} environment{ SONAR_HOME = tool "Sonar" } parameters { string(name: 'FRONTEND_DOCKER_TAG', defaultValue: '', description: 'Setting docker image for latest push') string(name: 'BACKEND_DOCKER_TAG', defaultValue: '', description: 'Setting docker image for latest push') } stages { stage("Validate Parameters") { steps { script { if (params.FRONTEND_DOCKER_TAG == '' || params.BACKEND_DOCKER_TAG == '') { error("FRONTEND_DOCKER_TAG and BACKEND_DOCKER_TAG must be provided.") } } } } stage("Workspace cleanup"){ steps{ script{ cleanWs() } } } stage('Git: Code Checkout') { steps { script{ code_checkout("https://github.com/LondheShubham153/Wanderlust-Mega-Project.git","main") } } } stage("Trivy: Filesystem scan"){ steps{ script{ trivy_scan() } } } stage("OWASP: Dependency check"){ steps{ script{ owasp_dependency() } } } stage("SonarQube: Code Analysis"){ steps{ script{ sonarqube_analysis("Sonar ","wanderlust","wanderlust") } } } stage("SonarQube: Code Quality Gates"){ steps{ script{ sonarqube_code_quality() } } } stage('Exporting environment variables') { parallel{ stage("Backend env setup"){ steps { script{ dir("Automations"){ sh "bash updatebackendnew.sh" } } } } stage("Frontend env setup"){ steps { script{ dir("Automations"){ sh "bash updatefrontendnew.sh" } } } } } } stage("Docker: Build Images"){ steps{ script{ dir('backend'){ docker_build("wanderlust-backend-beta","${params.BACKEND_DOCKER_TAG}","trainwithshubham") } dir('frontend'){ docker_build("wanderlust-frontend-beta","${params.FRONTEND_DOCKER_TAG}","trainwithshubham") } } } } stage("Docker: Push to DockerHub"){ steps{ script{ docker_push("wanderlust-backend-beta","${params.BACKEND_DOCKER_TAG}","trainwithshubham") docker_push("wanderlust-frontend-beta","${params.FRONTEND_DOCKER_TAG}","trainwithshubham") } } } } post{ success{ archiveArtifacts artifacts: '*.xml', followSymlinks: false build job: "Wanderlust-CD", parameters: [ string(name: 'FRONTEND_DOCKER_TAG', value: "${params.FRONTEND_DOCKER_TAG}"), string(name: 'BACKEND_DOCKER_TAG', value: "${params.BACKEND_DOCKER_TAG}") ] } } }

version: "3.8" services: mongodb: container_name: mongo-service image: mongo:latest volumes: - ./backend/data:/data ports: - "27017:27017" backend: container_name: backend build: ./backend env_file: - ./backend/.env.docker ports: - "31100:8080" depends_on: - mongodb frontend: container_name: frontend build: ./frontend env_file: - ./frontend/.env.docker ports: - "5173:5173" redis: container_name: redis-service restart: unless-stopped image: redis:7.0.5-alpine expose: - 6379 depends_on: - mongodb volumes: data: ------------------------------------- updatedbackendenv.sh #!/bin/bash # Set the Instance ID and path to the .env file INSTANCE_ID="i-030da7d31a1dbbffc" # Retrieve the public IP address of the specified EC2 instance ipv4_address=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[0].Instances[0].PublicIpAddress' --output text) # Path to the .env file file_to_find="../backend/.env.docker" # Check the current FRONTEND_URL in the .env file current_url=$(sed -n "4p" $file_to_find) # Update the .env file if the IP address has changed if [[ "$current_url" != "FRONTEND_URL=\"http://${ipv4_address}:5173\"" ]]; then if [ -f $file_to_find ]; then sed -i -e "s|FRONTEND_URL.*|FRONTEND_URL=\"http://${ipv4_address}:5173\"|g" $file_to_find else echo "ERROR: File not found." fi fi ------- -------------- updatefrontendenv.sh #!/bin/bash # Set the Instance ID and path to the .env file INSTANCE_ID="i-030da7d31a1dbbffc" # Retrieve the public IP address of the specified EC2 instance ipv4_address=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[0].Instances[0].PublicIpAddress' --output text) # Path to the .env file file_to_find="../frontend/.env.docker" # Check the current VITE_API_PATH in the .env file current_url=$(cat $file_to_find) # Update the .env file if the IP address has changed if [[ "$current_url" != "VITE_API_PATH=\"http://${ipv4_address}:31100\"" ]]; then if [ -f $file_to_find ]; then sed -i -e "s|VITE_API_PATH.*|VITE_API_PATH=\"http://${ipv4_address}:31100\"|g" $file_to_find else echo "ERROR: File not found." fi fi

```bash @Library('Shared') _ pipeline { agent {label 'Node'}

parameters { string(name: 'FRONTEND_DOCKER_TAG', defaultValue: '', description: 'Frontend Docker tag of the image built by the CI job') string(name: 'BACKEND_DOCKER_TAG', defaultValue: '', description: 'Backend Docker tag of the image built by the CI job') }

stages { stage("Workspace cleanup"){ steps{ script{ cleanWs() } } }

stage('Git: Code Checkout') { steps { script{ code_checkout("https://github.com/LondheShubham153/Wanderlust-Mega-Project.git","main") } } }

stage('Verify: Docker Image Tags') { steps { script{ echo "FRONTEND_DOCKER_TAG: ${params.FRONTEND_DOCKER_TAG}" echo "BACKEND_DOCKER_TAG: ${params.BACKEND_DOCKER_TAG}" } } }

stage("Update: Kubernetes manifests"){ steps{ script{ dir('kubernetes'){ sh """ sed -i -e s/wanderlust-backend-beta.* /wanderlust-backend-beta:${params.BACKEND_DOCKER_TAG}/g backend.yaml """ }

dir('kubernetes'){ sh """ sed -i -e s/wanderlust-frontend-beta.*/wanderlust-frontend-beta:${params.FRONTEND_DOCKER_TAG}/g frontend.yaml """ }

} } }

stage("Git: Code update and push to GitHub"){ steps{ script{ withCredentials([gitUsernamePassword(credentialsId: 'Github-cred', gitToolName: 'Default')]) { sh ''' echo "Checking repository status: " git status

echo "Adding changes to git: " git add .

echo "Commiting changes: " git commit -m "Updated environment variables"

echo "Pushing changes to github: " git push https://github.com/LondheShubham153/Wanderlust-Mega-Project.git main ''' } } } } } post { success { script { emailext attachLog: true, from: 'trainwithshubham@gmail.com', subject: "Wanderlust Application has been updated and deployed - '${currentBuild.result}'", body: """

Project: ${env.JOB_NAME}

Build Number: ${env.BUILD_NUMBER}

URL: ${env.BUILD_URL}

""", to: 'trainwithshubham@gmail.com', mimeType: 'text/html' } } failure { script { emailext attachLog: true, from: 'trainwithshubham@gmail.com', subject: "Wanderlust Application build failed - '${currentBuild.result}'", body: """

Project: ${env.JOB_NAME}

Build Number: ${env.BUILD_NUMBER}

""", to: 'trainwithshubham@gmail.com', mimeType: 'text/html' } } } }

* --- --- \- * **Now, go to github repository and under Automations</mark> directory update the instance-idark> field on both the updatefrontendnew.sh updatebackendnew.sh with the k8s worker's instance id** ![image](https://private-user-images.githubusercontent.com/121779953/358123023-3cb044b4-df88-4d68-bf7c-775cf78d5bf2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg2ODEzODMsIm5iZiI6MTc0ODY4MTA4MywicGF0aCI6Ii8xMjE3Nzk5NTMvMzU4MTIzMDIzLTNjYjA0NGI0LWRmODgtNGQ2OC1iZjdjLTc3NWNmNzhkNWJmMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwNTMxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDUzMVQwODQ0NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yZDk3NzQ4NTIzMjkxOTk1MWU2OGQwNzZiNGExZjdkOGI4N2M1NjZmNGY3MWY3YzFhYzAxNTJmYzRiY2QxNDZmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.JhLdvfvLd_4RMG5mgJaqvrPDQ7fwMh2AROgFtnLUZ6g align="left") * **Navigate to Manage Jenkins --> credentials and add credentials for docker login to push docker image:** ![image](https://private-user-images.githubusercontent.com/121779953/358123993-1a8287fc-b205-4156-8342-3f660f15e8fa.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg2ODEzODMsIm5iZiI6MTc0ODY4MTA4MywicGF0aCI6Ii8xMjE3Nzk5NTMvMzU4MTIzOTkzLTFhODI4N2ZjLWIyMDUtNDE1Ni04MzQyLTNmNjYwZjE1ZThmYS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwNTMxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDUzMVQwODQ0NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01ZDVmMmQwMDAxYWZlMmQwNGMzMzgyNmQ3NDRhNzBmMTI2NWU4ZTlkYjJjMjA5NmNlYWY0YWYzOTIwMjQwMzIwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.HRF65YDSGNiE-NBxKHqp_9QK8R9OP8DblRK3VwxNztM align="left") * ae sirf CI hey ae success hua then only CD karo - samja * Now create a pipeline * **Create a Wanderlust-CI pipeline** ![image](https://private-user-images.githubusercontent.com/121779953/358123234-55c7b611-3c20-445f-a49c-7d779894e232.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg2ODEzODMsIm5iZiI6MTc0ODY4MTA4MywicGF0aCI6Ii8xMjE3Nzk5NTMvMzU4MTIzMjM0LTU1YzdiNjExLTNjMjAtNDQ1Zi1hNDljLTdkNzc5ODk0ZTIzMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwNTMxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDUzMVQwODQ0NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yZTBmY2M5ZWUzYTZhOTg1NGJkNmZkZDNmYjZjYzE5MTFiZDI4YTFlYzBhZDBlMzkxZjlmMGYzYTRmYmQ4ODhhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.fLszV77WOTj7WjSrRHkGio7XwDap_QO0vKEdmkkjXLY align="left") * **Create one more pipeline Wanderlust-CD** ![image](https://private-user-images.githubusercontent.com/121779953/358482252-23f84a93-901b-45e3-b4e8-a12cbed13986.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg2ODEzODMsIm5iZiI6MTc0ODY4MTA4MywicGF0aCI6Ii8xMjE3Nzk5NTMvMzU4NDgyMjUyLTIzZjg0YTkzLTkwMWItNDVlMy1iNGU4LWExMmNiZWQxMzk4Ni5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwNTMxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDUzMVQwODQ0NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05NGEyZjVlODBkZjYxMmZhMzE3ODIwZGUyYTY5ZWM5NTc2ZGVjN2YzOTlmZGZjYTgyODU5OWQ2NmE1MGI5MmRhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.SBmoMO60Jg85BG124Hvt5GPrWc6v1k4ywl9atJUhGwU align="left") ![image](https://private-user-images.githubusercontent.com/121779953/358482407-ac79f7e6-c02c-4431-bb3b-5c7489a93a63.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg2ODEzODMsIm5iZiI6MTc0ODY4MTA4MywicGF0aCI6Ii8xMjE3Nzk5NTMvMzU4NDgyNDA3LWFjNzlmN2U2LWMwMmMtNDQzMS1iYjNiLTVjNzQ4OWE5M2E2My5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwNTMxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDUzMVQwODQ0NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lYTkzODU2OTU3Y2IzNWQxOGNmNGNjNDcwZGM2YjQyNmQzZjJkOWNjZmIzOTFiNjc0YjVkMzJkOGEwNzhhMTM2JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9._qWTqR5JeGvhCEXB6njWMOyHmlwhP0ZJ6Pf9sqhwn9k align="left") ![image](https://private-user-images.githubusercontent.com/121779953/358482633-46a5937f-e06e-4265-ac0f-42543576a5cd.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg2ODEzODMsIm5iZiI6MTc0ODY4MTA4MywicGF0aCI6Ii8xMjE3Nzk5NTMvMzU4NDgyNjMzLTQ2YTU5MzdmLWUwNmUtNDI2NS1hYzBmLTQyNTQzNTc2YTVjZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwNTMxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDUzMVQwODQ0NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1jZjhhMDk3OTYzMDUxMzY0MzljNTA0YTFmOGNjZTQ1NGYxMWY3N2Y1YWM4YmMyYzIwOWZhNmU4YWNiNzJiOTlmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.i36twuR057dizohtoGK-DxGZY8TK8cWIRKNJt-lGpxQ align="left") * deploy kaisa karega- argocd - aapka ae jo cluster hey us cluster ko argocd key sath connect karna padtha hey * sabsey pehley * argocd login url —usernamme admin → y → password * argocd cluster list * **Provide permission to docker socket so that docker build and push command do not fail (Jenkins Worker)** ```bash chmod 777 /var/run/docker.sock

Go to Master Machine and add our own eks cluster to argocd for application deployment using cli

Login to argoCD from CLI

argocd login 52.53.156.187:32738 --username admin

Tip

52.53.156.187:32738 --> This should be your argocd url

Check how many clusters are available in argocd

argocd cluster list

Get your cluster name

kubectl config get-contexts

Add your cluster to argocd

argocd cluster add Wanderlust@wanderlust.us-west-1.eksctl.io --name wanderlust-eks-cluster

Tip

Wanderlust@wanderlust.us-west-1.eksctl.io --> This should be your EKS Cluster Name.

Once your cluster is added to argocd, go to argocd console Settings --> Clusters and verify it

Go to Settings --> Repositories and click on Connect repo

Note

Connection should be successful

Now, go to Applications and click on New App

Important

Make sure to click on the Auto-Create Namespace option while creating argocd application

Destination select - cluster url and select namespace

Congratulations, your application is deployed on AWS EKS Cluster

Open port 31000 and 31100 on worker node and Access it on browser

:31000

Email Notification

# Meanwhile we will setup prometheus and grafana

sab sey pehle ae samjo helm kya hotha hey - helm ek package manager

agar mujhey patha hey eks kya hey - eks is a kubernates service by aws webservice - uskey andar jo jo components humko patha hey

jo humko pata hey toh baar baar kiun edit karey - kya mai promethus .yaml mey ae saare configurations daldunga

helm mey ae saari configuration mey already dali hui hey , eks key components already helm mey hey

aapko helm install karna hey agar ./get-helm.sh run karenge na toh install hojatha hey helm

once pipeline deployment is done

jenkins mey ek iam role banana bhulge - iam - create a role - aws service - ec2→ specific role - admin access → ec2-wanderlust-role

is role ko master node pey attach karne wala - actions - security - modify iam role - select the iam role - and attach it .

now i will run v1.1.

kya aap grafana through dashnoards mey - merey k8s key namespaces key pods hey uska dashboard dikha sakthey ho

How to monitor EKS cluster, kubernetes components and workloads using prometheus and grafana via HELM (On Master machine)

Install Helm Chart

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3

chmod 700 get_helm.sh

./get_helm.sh

Add Helm Stable Charts for Your Local Client

helm repo add stable https://charts.helm.sh/stable

Add Prometheus Helm Repository

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

Create Prometheus Namespace

kubectl create namespace prometheus

kubectl get ns

Install Prometheus using Helm

helm install stable prometheus-community/kube-prometheus-stack -n prometheus

Verify prometheus installation

kubectl get pods -n prometheus

Check the services file (svc) of the Prometheus

kubectl get svc -n prometheus

Expose Prometheus and Grafana to the external world through Node Port

Important

change it from Cluster IP to NodePort after changing make sure you save the file and open the assigned nodeport to the service.

kubectl edit svc stable-kube-prometheus-sta-prometheus -n prometheus

Verify service

kubectl get svc -n prometheus

Now,let’s change the SVC file of the Grafana and expose it to the outer world

kubectl edit svc stable-grafana -n prometheus

Check grafana service

kubectl get svc -n prometheus

Get a password for grafana

kubectl get secret --namespace prometheus stable-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Note

Username: admin

Now, view the Dashboard in Grafana

Clean Up

Delete eks cluster

eksctl delete cluster --name=wanderlust --region=us-west-1

Docker in One Shot

Vaseem Shaik — Thu, 29 May 2025 16:30:04 GMT

Agenda:

intoduction to Docker

Docker hotha kya hey , jabbi samajlo jabbi aapko ek cient hey , client key pass laptop hey uskey server pey deploy karna hey , mainey devolop kiya mera own machine machine its working me , but in client location its not working

docker sek template banakey image create karkey run kartha hey application on containers - jo b requirements , libraries sab ek image mey rakhey ship karthey hey is image ko kisi b computer mey chala sakthey hey

Docker ek open source tool hey

Docker aaya .cloud key company thi ae problem ai , inhoney is cheez ko containarised kardiya aur open source kar diya , docker ka janam idhar hua

2013 , publisher hey pycon , 2017 mey CNCF vaha pey ae pauchee , vaha sey bohot log use karrey

Why we use dokcer : kabhi kabhi aapkey computer mey chalri , isliye ki , kuch libarires , versions ,kuch tools isliye vo cheez chalri hey ,dusre computer mey ae cheez nahi ho isliye aapko container or vitua environment banana padtha hey , aur usi virtual environment key vajhasey dusrey computers mey b ae chez chalthi hey

kya ae docker vm jaisa rehtha hey

vm vs docker -

irtualization hotha hey usmey virtual machines hothey hey

containerization mey docker , container d , podman containers ho sakthey hey

jo vm hothey hey na , ae ek cheez use karthey hey hypersior , hypervisor aapkey o.s sey jagah resources allocate karletha hey uskey uppar o.s chalatha hey

vm mey khudka operating system hotha hey , vitualizatinon hotha hey bohot load hotha hey multi o.s

containerization ek tool use kartha hey docker engine - ae khud ka o.s use nahi rakhtha vo host key o.s ko use karletha hey

jo apna container bantha hey light weight bantha hey

Daigramatically :

Docker Architecture kya hey :

Docker kaisa kaam kartha hey → docker mey three important cheez hey

docker engine : dokcer application container engine iskey uppar containers chalre

docker daemon : docker d or docker daemon - ae containers ko manage karthi hey is docker d key ander aur ek cheez rehtah hey container d - ae ek tool hey

ae containers kaisa banange , kaisa instructions denge -ae docker cli key through chalthey hey vo commands docker daemon ko paunchthey hey , jo ki background process hothi hey ae cotainer dki bolthi hey bhai ae chalado ae karado akrkey

docker client: jaha key docker key jithney b cheeze heyna dekh sakthey hey , toh docker client connect kartah hey api → ko toh aisey cheexe hothey hey backend sey communicate kar sakaey , aur jo b hey dikhadega

iskey backgroup mey docker daemon and cli khel chalare hothey hey

# install docker :

sudo apt-get update -y

sudo apt-get install docker.io - docker enginer , dokcer daemon , docker cli , docker client , containerd , sab miljayega

sudo systemctl status docker

whoami - docker ps - > sudo usermod -aG docker ubuntu

newgrp docker # is command sey aapka group refresh hojayega

docker ps - # docker cli sey docker d ko command bheja containers list dikhavo karkey

docker images:

- docker images kya hotha hey

- aap kabhi school yah colleg emey kabhi chitte banayethey kya - ek paper mey chotey chotey likhey huey rehthey hey - us chethey sey apni answer script banathey ho - ithna sa chitt apass laratha hey

- ae hi chitta apkey dost ko dedo vo bhi pass hojayega

- ae jo chitta hey vo ek image hey - is image chittey sey banathey hey vo aisa samajlo containers

toh us image ko aap kahi pey b daldo - us cheez ka container bana sakthey hey right

vo image bani or likhi jathi ey dockerfile sey , bhaiyya dockerfile sey image - image sey container ban jathi hey lekin image kya hey

image aek aisa blueprint hey jismey saare steps likhey huey hey , kya aap ek application ko run karnekey liye kya kya chahiye , kaisa environment chahiye , kya kya hona chahiye us image ko directly banakey run kar sakey - its like blueprint

docker file mey ek instsructions likhey honge , us instructions key basis pey image banthi hey , us image ko replicate kar sakthey hey kithne b containers bana saktehy hey

lets say mai hun, mainey bohot badiya sa notes banaya , us notes sey docker in one short sey ek video or ek image bangayi , aap dekhoge ho sakeytho aap key doston ko bhejogey ae conatiner hothi hey

Agar nutshell b bataun - dockerfile mey ek instructions hothey hey aplications ko jo b required cheeze - is docker file sey image banaega - docker image sey bantha hey docker container

pre-build images ko b la sakthey hey

docker login karlo - docker login sey aapko docker hub mey login karna padtha hey

account settings → PAT → Generate a new token → read write access

Login suceeded - docker hub aek aisi jagah hey - docker key public saarey images idhar rakhi hothi hey

#mujhey docker image pull karni hey kaisey oull karunag -

docker pull hello-world

docker images - list of all docker images

is image ko kya karna hey , is image sey bantha hey container

docker run hello-world

docker pull mysql # mysql ko run karneykey liye jo image bani hogi vo bhi download kardega

mysql ek server hotha hey login honekey liye usernname pasword dena padtha hey

docker run -e MYSQ_ROOT_PASSWORD=root mysql

docker ps - mysql ka container chalra hoga 0 is container ka b lifecycle hotah hey → chalrahey , chalrahoga , rukh jayega , phir wapas sey restart , phir kill hojayega .

docker stop conatinet_id → vo container ko rok dega

agar mujhey background mey chalana hey toh -d = detached mode mey use karo

Mainey jo b pull kiya vo publically available image ley , lets say merey pass ek java ka project hey

us java ka projetc ka conatiner mujhey banana hey iskey liye mujhey dockerfile likhni hogi - chala dockerfile banathey ho

shubham loundhe - simple-java -clone it - as a devops engineer aapko pata chalna hey ki is java ko run karnekey liye dockerfile likhni aani chahiye

vim Dockerfile

# pulla abase images whcih gives all required tools and libraries FROM openjdk:17-jdk-alphine # create a folder where the app code will be stored WORKDIR /app # copy the source code form your host machine to conatiner COPY . . COPY src/Main.java /app/Main.java # Compine the application code RUN javac src/Main.java # Run the application code CMD ["java" , "Main"]

Docker file hotha kya hey , aisa samajlo kuch instructions hothey hey us instricions key vajaheysey aapka applicaton run hoga

instrictions kya hotah hey us key andar - application code , libraries , jo o.s use akrni hey uskey bharemey , frmework , tools , packages ae sab rehni chahiye

ae box ko bananey key liye dockerfile mey istructions dena hey

kabhi maggie banaye - maggie banaye instructions rehthey hey right

step1. kadai , step2. pani step3. masala, step4.maggie step5. gas - to build recipe for maggir , agar tum ae recipe dedi vo us recipe sey maggie banayega

ae java application run karna hey , java , mujhey first ek environment hona hey aur uskey andar java honi hey

Base image: FROM openjdk:- > image tag kya hotha hey - image ka version hotha hey

issey ek environment miljatha hey

mujhey ab working directory banana hey , workdir container mey folder bantha hey uskey andar merey application code copy karunag

RUN javac main.java #mujhey agar koi file or command run karni hey mai RUN command use kartha hun

Jabbi aap ek line likhthey hey - jabb ek image banthi hey har ek line ek layer hothi hey jissey vo image hori hey

ae jo intermeditae container ko liye lines hothey hey

jo cmd hotha hey ek special command hotha hey aapka conatiner bun aye us container ko run karnekey liye - CMD use karthey hey

pani oiney straw chahiye right - vaisa hi cmd - command container run karne use akrthey hey

so ae command nahi likhog tabhi container banega , agar container likhre ho container banneykey badh execute hojayega

is comamnd ko overwride kar sakthey ho - while running the octainer

is ka b bada bhai hey - ENTRYPOINT - entry point overwride nahi ho saktha - ae aisey command hey intermediate container bananey nahi hotha - container mey command inject hoyega

docker build -t java-app . location - steps are layers - ab aapki java name ki image ban gayi

docker run java-app

clone python project - flask-app-ecs → app.py file kuch toh code hey api’s detha hey

git clone url vim Dockerfile FROM Python:3.7 WORKDIR /app COPY . . RUN pip install -r requirements.txt ENTRYPOINT "python","run.py"

- docker build -t flask-app .

docker run -p 80:80 flask-app

jaisa mainey aapko baytaya ae aapka o.s (host) hey iskey pass resources hothey hey jo docker engine share karletha hey aur containers ko chalaletha hey jo container key andar app chalra hey port number 80 sey chalra hey aur appka 0.s ka port dono ko bind karana padtha hey

us cheez key liye publish kara na pastha hey -p host:container port

is aapko background mey chalana hey background mey run kardo -d

docker run -d -p 80:80 flask-app

docker ps - karkey dekhunga mera app chalra hey yah nahi

port enable karni chahiye dekhnekey liye - publically open karna padtha hey

agar aapko lagra hey aapka app time lera hey toh logs check kar sakthey hey docker key andar kya chalra hey - docker logs container_id

har bar loga vala command chalana padra - aisa kuch hey kya ki aapka container atatch hojaye is screen per ki ssarey logs realtime dekh pau - docker attach cont_id

aap cont stop kar sakthey hey - docker stop cont_id

jaisa aapne stop kiya vaisa hi start kar sakthey hey - docker start cont_id

aapka mysql b chalra , kya mysql ka data base hey uskey andar ghuskey access kar saktah hun ? - > yes → docker exec -it cont_id bash

mysql -u root -p root

show databases;

create database devops;

exit

docker ps - running containers dikhthey hey

docker images - aapkey pass jithney b images vo dikhthey hey

mujhey ubuntu chalana hey - aisa container chalana hey hamesha running mey ho mac mey ubuntu chalana hey

docker run -itd ubuntu

Docker Networking :

Docker ki networling kya hotha hey

lets say aapka aek computer hey is computer mey ek docker ka conatiner chalra hey , aapkey pass aur ek docker container chalra ek flask app aur mysql ka container hey - ae flask app ka container hey ek isolated , aur vo b isolated - ae flask mysql sey bath nai kar saktha , mysql flask sey bath nahi kar saktha

agar aapko in dono key beechmey communication karwwani hey toh aapko ek network chahiye hoga

isliye aathey hey concept aapka docker networks

docker networks 7 prakar key hothey hey

Host network - ki aapki docker ocnt netwrok hey aur aapkey host machine ki network hey vo same rahega

toh aapka port number 80 host mey chalra hey aur aapne docker network mey host dala hua hey docker cont ka b port number 80 chalega

default bridge : jaisey aapka ae cont hey aapka ae host hey - , is host sey is cont pey network bananey key liye aapko ek pull bandna padthey hey - by default dethey hey

user defined brige (custom_bridge): khudna network create kar sakthey hye

None: matlab is docker conainer mey koi netwrok nahi , na ussey koi bath kar skathey hey , na vo kisi sey bath kar skatha hey , completely isolated , na inetrnet chahiye container ko ae netwrok ded-

macvlan : macvlan basically mac address , 2 mac address pey communicate kar saktha hey - jab docker swarm cluster rehtah hey thab use kar sakthey

ipvlan

overlay

ae jo b 7 hey isko drivers bolthey hey

ae jo last three ja docker swarm cluster rehtah hey thab use karthey hey

docker swarm kya hotah hey , abhi is laptop pey docker cont banaye - agar 50 las mey docker ocntainer pey banavo aur uso sabko ek sath chalav

cluste boltheyhey , aajkal kubernates use karthey hey uskey liye ae last three use karthey hey

frst 4 humko important hey

kaisey padthey hey docker network - docker network ls

abhi three network aapkey samney hey

docker network create mynetwork -d bridge

iska use hoga ek project mey- jab hum two tier ka project karenge thab - samjo ek flask aap and mysql conatienr chlara - ae jo flask app hey ae ek docker cont hey aur mysql b - ab dono jo docker conatiner independently chalre mujhey koi problem nahi aari - agar merko is dono ko communicate karani hey toh use karthey hey docker networks

#connection of two tier application

go to github - clone two tier flasp app- clone it

ab is app mey app.py , kya ae application kissi data base ko connect kartha hey - > ismey likha hua configure my sql from environment variables - jo envi ronment variales hey pass karna padtha hey ae application run karwanekey liye

docker run -e → issey environment pass karthey hey

vim Dockerfrile

FROM python:3.9-slim WORKDIR /app RUN apt-get update\ && apt-get upgrade -y \ && apt-get install -y gcc default-libmysqlclient-dev pkg-conf COPY requiremnets.txt RUN pip install mysqlclient RUN pip install --no-cache-dir -r requirements.txt COPY . . CMD ["python","app.py"]

docker build -t two-tier-backend .

jo mysql hey already usiko hi utilise karlunga

kya kya variable pass karna app.py mey dekhlunga

docker run -d -e MYSQL_HOST=mysql -e MYSQL_USER=root -e MYSQL_PASSWORD=root -e MYSQL_DB=devops -p 5000:5000 two-tier-backend:latest

docker ps -a - simple nahi dikhara matlab exit hogaya karkey

docker logs cont_id - logs check karey

kiun chalra nahi matlab - is flask network mey mysql ka cont hi mil hi nahi raha

docker stop mysql_id & docker rm cont_id

mai ab ek mysql cont banaunga ussey pehle ek network create karlunga - two-tier drive hey bridge

ae network dono containers ko dedunga

docker run -d --name mysql --network two-tier -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=devops mysql

docker run -d --network two-tier -e MYSQL_HOST=mysql -e MYSQL_USER=root -e MYSQL_PASSWORD=root -e MYSQL_DB=devops -p 5000:5000 two-tier-backend:latest

docker ps - two-tier-backend flask , mysql both are running if i check logs = > docker logs cont_id

docker network ls - ae jo create kiys → docker network inspect cont_id - dono ekhi network toh dono bath kar skathey hey ek sey ek

ab enable kardo 5000 port public to access

docker exec -it mysql_contid bash

mysql -u root -p root

shwo databases;

use devops;

select * from messages;

Docker volume and storage:

ae jo hum log dekha data base mey data stored hey ae aara application sey - agar aap mey data dalenge toh db mey reflect hora

agar mainey database valey container ko mysql cont ko restrt kardiya kya hoga

docker restart mysql-contid

docker stop and start refresh hoga - agar mai jakey sql cont mai ab jakey dekhunga toh jo data previously tha vo hey abhi bhi

agar mai restart ki jagah container ko delete kardun toh - docker stop cont_id & docker rm cont-id

docker ka same container vapas run kartha hun - jab mai isko refresh kartha hun saara data ja chuka hoga delete ho chuka hoga

ab ae ek problem hey , ae docker volume and storage -→ae kya hotah hey ae aek tariqa hotah hey docker key through jithey b containers hey uska data persist kara saktha hun

jissey ki galthi sey delete hojai , restrt , stop and start karey mera data persist rahey

to for ex - ae aapka mysql is ae jo cont iskey andar ssara data hey , agar cont delete hoga toh cont delete hoga

iska jo data hey usko bind karlethey hey apney host key sath ,apney host hotah hey filesystem hotha hey , usmey ek path folder banalethey hey cont key data key sath , vo jo hotha hey concept bolthey hey volume

aur is cheez ka command hothah eyy -v or docker volume

docker volume ls → karunana jo b volumes hey dikh jathi hey

mai khudki volume kaisa create karunga - aap docker volume create mysql-data

ab ae jo volume hey ae kounsey jagah pey rahegi - docker inspect myvolname - ae /var/kib/docker/volumes/mysql-data/_data

ab mujhey apney container ko is path key sath bind karna hey - > mysql cont abhi key liye delete kardunga & docker rm cont_id

ae jo mysql -vol dithi mai mysql-cont sey bind kardunga

docker run -d --name mysql --v mysql-data:/var/lib/mysql -network two-tier -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=devops mysql

mysql ka data stored in /var/lib/mysql

jo b data mai flaskapp sey banauga vo data delete nahi hoga

docker stop cont_id docker rm cont_id -→ ab mai isko delete kardunga toh ky mera mysql ka data b udega

docker volume inspect myvol

sudo su —> cd path = ist ther emy saara data secured hey ,- mai same volume say sam container create karunga

dokcer restart flask_app_id —> as it is stored .

# data persist ka aur ek tarika hey kya -→ aap kya karko bahar aajav - ek volume name ka folder banado - cd volumes→ mysql —> ab aap aap key two-tier flask app mey jakey - docker container ka rehtha ha mysql vala - docker stop mysql & docker rm mysql

#VOLUMES 2 TYPES KEY HOTHEY HEY

1. named based volume:

path volume

mysql

cd volumes →mysql → saara data backup

agar mai dokcer run mey -v rakhey jo b folder create kiya agar vo folder path dedunga toh b container chalega - isko bolthey path based volume

you can save your data with docker volumes.

Docker Compose:

abhi mainey bohot saara manuall kaam karra hun like , mai baar baar ek two tier flask app mey jatha hun , docker ps kartha hun , agar docker conatiner ko delete karna hey , docker stop or docker rm - > manually karra hun

as a devops engineer manually nahi karna , automate karna hey

docker compose automate karnemey bohot help kartah hey

docker compose mey ek configuration file rehtha hey vo aapko bohot madad kar sakthi hey

yet another markup language - > jaha pey key:value pey cheezey pass kardethey -

ab docker compose naame kiun diya , ek aur ek sey zyada containers create karney mey help kartha hey

jo b hum log ney abhi thak kiya docker build ,docker run , docker network , docker volume ae sab aek hi sath mey file sey kar deta hey

docker compose mey multiple docker containers bana sakthey ho, simple ka funda , bananey ka tarika hey combination of build , run , network volumes

docker compose ka version “3.8” - ae syntax ka version hey

aur aap services daloget - i mean kithney containers karkey bata sakthey hey

iskey badh docker build use karthey hey na image lekey -idhar direct image lethey hey - image:→phir environmnet kuch dena hey idhar dey sakthey hey

phir volumes - mysql-data:/var/lib/mysql

networks karrethey - two-tier

port - “3306”:”3306

abhi flask na image sey nahi hotha , build sey hotha hey - aur iskey liye context lagtha hey - context hey docker file kaha rakhi hui

phir container_name:, ports: “5000”:”5000” , ab environments use kartha hey jo flask environments mey jo chahiye vo , phir networks, vo

volumes : jo create kiye uskey name daldo , networks: iska network ka name b daldo

version: "3.8" services: mysql: user: "${UID}:${GID}" image: mysql:5.7 container_name: mysql environment: MYSQL_ROOT_PASSWORD: root MYSQL_DATABASE: devops MYSQL_USER: root MYSQL_PASSWORD: root volumes: - ./mysql-data:/var/lib/mysql - ./message.sql:/docker-entrypoint-initdb.d/message.sql networks: - two-tier restart: always healthcheck: test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-uroot", "-proot"] interval: 10s timeout: 5s retries: 5 start_period: 60s flask: build: context: . container_name: flask-app ports: - "5000:5000" environment: MYSQL_HOST: mysql MYSQL_USER: root MYSQL_PASSWORD: root MYSQL_DB: devops depends_on: - mysql networks: - two-tier restart: always healthcheck: test: ["CMD-SHELL", "curl -f http://localhost:5000/health || exit 1"] interval: 10s timeout: 5s retries: 5 start_period: 30s volumes: mysql-data networks: two-tier:

is tarah sey aapney badiyasa - docker file banaya → aapko is system mey docker compose ko install karna padega

sudo apt-get install docker-compose-v2

docker compose up

idhar mey dekho , yaha pey problem , mysql abhi thak bana nahi hey , two-tier flask container mysql sey pehley hi bangaya hey ae toh gadbad hey , ctrl c gracefully stop kardo

ab mai vim docker-compose.yml → ae jo mera flask vala container hey → depends-on : -mysql , jab thak mysql banjatha jab thak mera flask vala banna nahi chahiye .

docker compose up - kehra hey , abhi bhi khera , usney mysql ka container pehley hi bana diya , start kardiya ,entry point bandiya jaisi bangaya usney two tier ko start kar diya , problem kaha aathi hey pata hey , mysql ka pehley ban jatha hey , uskey two-tier flask bantha hey but mysql ka container accept kartha na ready for connections vo badh mey kartha hey , matab jab thak mysql sahi tarikey sey ready nahi hojatha thab tha usko pura ready nahi bol sakthey

isliye aapko ek cheez lagthi hey usko bolthey hey health check , docker compose mey mysql container key sath healthchecks: dalna padthey hey - ismey test key commands dal sakthey hey

ki fro ex: gara mysql mey mai ping kartha hun localhost ko apney passwords key sath , jab thak local host ka ping nahi hotha , thab thak vo retray kartah rahey 5s and 10s once

vaisa hi flaskapp mey b health checks dal sakthey hey

agar aapka health check pass nahi huye restart:kardo always

har baar restart karo jab thak test checks pass nahi hojathey , aur health check baar baar 10s interval mey 5 baar retry karthey raho - ab isko run karthey hey .

just a single command container up hojayega , ab mai detached mode mey run karunga - docker ps

This is how docker compose works.

Docker Registry :

Aisa samjo ae jo docker images hey - tow-tier flask or saare images local pey hye , isko kisi server or kisi repository or registry mey store karna hey jaisa ki docker hub , is image ko properly tag karna padega push karna padega ,

docker system prune: jithne b aapkey system unsed volumes , betworks , containers , dangling images sab delete kar detha hey

docker images - docker rmi -f image_id

multiple images and containers hey toh mai - -> docker images -aq → docker rmi -f $(docker images -aq)

docker compose down - containers delete stop kar detha hey aur remove kar detha hey

docker registry - jaisey mainey bataya - two-tier flask image hey local mey , merko is image ko repo mey paunchau -jo ki dokcer hub ki repo - docker login -u -p passwor

ab humko image ko tag karna hey , image tag ka matlab - jo purani latest image hey usko rename kar sakthey hey - agar aapko local pey image store karna hey normal name kaafi hey

agar aapko docker hub pey image rakhni ,, aapka jo b username hey vo us imge name key pehle aani chhaiye

docker image tag two-tier-flask:latest vasee143/two-tier-flask:latest

docker push vasee143/two-tier-flask:latest → ae local sey docker mey mera image pus karra hey

mulsti-stage Docker builds

docker images - karkey dekho ae jo image hey - jo humney banaya tha

flask app -ecs ka us image jab b humne build kiya python 3.7 base image utata hey - vo bohot badi image hothi hey , jo mera final image bantha hey na - vo b badi image bankey aatah hey

jab b mera application bohot bada hojayega 0 jab image ka size bada hojayega ae concern hey , problem hey

dokcer images - ae 1.1GB ki hey - ae multi stage docker kya karega aapki image ko stages mey distribute kar detha

aapki image banthi hey - dockerfile sey - uskaimage 1gb - uski ocntainer b bada hotha hey

ek baar aapkey dockerfile mey kya kya cheeze rehthey dekhthey hey

mujhey is size ki reduce karna hey toh merey dokcerfile mey base image hi badi hothi hey toh mai isko reduce kar saktha hun

kabhi kabhi full base images , libraries install karnekey liye jaroori hothi hey

sirf libraries key liye jaroori hothi hey run karnekey liye nahi

# stage1 base image 994mb FROM python:3.7 AS builder WORKDIR /app COPY requirements.txt . RUN pip install -r requirements.txt #---- #stage 2 base image 400mb FROM python:3.7-slim WORKDIR /app COPY --from=builder /usr/local//lib/python3.7/site-packages/ /usr/local/ /lib/python3.7 COPY . . ENTRYPOINT ["python","run.py"]

ae jo base image hey - isko aek alias name dey detha hun → ae sir libraries and packeges build hogi is stage sey - mai nayi stage banaunga uska base image - vo chothi base image use karunga

slim vali jiski size kam rehthi hey sirf run karney key liye

aur wrokdir create karo stage 2 ka - aur packages ko copy karlo from stage 1 then copy karlo code phir run kardo

docker build -t flask-app-mini .

Humney multi stage build sey 1.gb sey 140 mb thak compress kardiye - that is the power of caching and multistage build hey.

Monitoring and logging in docker

toh jaisey apney kiya docker ps - lets sey aapka container chalana hey toh jo humney mini appp banya na usko hi run karkey dekhthey hey

docker run -d -p 80:80 flask-app-mini

ip:80 - this should run - lets say isko debug karna hey

docker logs cont_id - logs dikhege idhar , aap chaho aur ek cheez kar sakthey hey ,

jithney b logs hey ek file mey redirect kar sakthey hey - nohup - kisi b commands ko background mey run arkey file mey store kar detha hey

nohup docker attach cont_id &

ls

cat nohup.out - aap loga collect kar paarey ho

mai jo b karunga ip:80/thanos - ae servers key logs and monitors kar sakthey hey

Jis tarah sey docker containers mey maza aatha hey kubernatest mey b aatah hey

Orchestrating docker with kubernates (Introduction)

Toh sabsey pehley samjo k8s kya hotha hey - aap docker padthey padhey aapko patah chala ae conecpts hey karkey

docker hum log directly production mey use nahi karthey - kiunki jaisey abhi mainey kiya docker ps - simple sey mai docker stop kardunga na downtime hojayega - agar docker rm kardunga uska data application delete hojayega

docker containers crash honekey chances zyada hotha hey - production mey isiliye hum conts ko individually nahi chalathey production mey

isliye containers ko orchestrate karthey hey

-Kubernates aek orchetsration tool hey jo docker cont ko manage kartah ehy

ek docker cont kill hogaya , crash hogaya toh dusra container up hojaye

agar ek docker cont bohot storage and ram zyada lera hey , us storage and ram ko allocate kardo memory jissey ki kuch limits mey hi container chale

jab amazon use karrey hey - jab big billion saley hotah hey - thab bohot zyada load hotah hey - agar crash hojatah ehy thab auto jheal kar detha hey - agar load zyada load aatah hey k8s scale kar detha hey based on criteria , background mey containers hi chalre hothey hey , bus is tarah sey chalre hotey hi uskey pass auto scaling aur auto healing capaticy sey chalre hothey hey

Kubernates mey kuch cheeze hothey hey like

pods: pods aisa hothey hey ek unit hey iskey andar containers run hothey honge , ek aur multiple pods b run hothey hondey

agar merko der saare pods chahiye (collection of pods(replicas chahiye) humko deployment chahiye - ae jo deployment rehthi hey frontend , backend , data base ki alag banathey hey - un deployments ko ek dusre sey bath karnekey liye network jo chahiye hotha hey aap service sey dethey ho - (connect the deployments to the users)-

ingress - usmey b routing karni hey ingressuse karna padtha hey

Projects:

project1: 3 Tier Application with Docker Compose

project2: Deploying a web application with Nginx and mysql

PROJECT2:

Dosto agar project ki bath karey nginx with mysql - nginx ae industry standard reverse proxy tool hey

now we are gonna pair up nginx and mysql

project hey - web app hey (django mey likhey hey ) vo nginx ser server hoga - djnango ka backend hey - vo data base sey connect hoga mysql ka

aapko anana hoga -django , django ka application , vo connetc hoga mysql database sey ae pure application sever hoga nginx sey

ae b kind of three conatiners use karney vala hey

nginx cont , django cont , aur mysql cont

ae mysql cont hey ae djnago cont though bat karega ismey b network lagana padtah hey

nginx hotahhey , django or web applications hothey hey vo port pey chlathey hey , ab 800 key andar aur b andar routing chalri ho , vo routing internally route hojai externally port number dena na padey isliye ek reverse proxy server hota hey

proxy matlab aap unko ek proxy pass dedo agar aaka url localhost:8000 aaya , agar aapka url localhost/app or localhost/home aaya tohi internally yaha redirect kardo

mainey apney website mey dala - ip/ daldiya nginx sever key routing mey aisa likha hoga agar / hey django pey redirect kardey agar /db hey mysql ko redirect kardey

go to github - django-notes-app - ae jo aap hey - is appko hum end to end build karenge

clone karlo - ismey kya kya cheeze hey samajthey hey

ismey my notes hey ae frontend hey - frontend hey ae already build bani hui files hey - iska jo important hey backend hey (notesapp( backend hey jo django mey likha hua hey

settings.py ae jo file rehthi hey sab configuration rehthi hey

templates key andar mynotes/build - ae frontend key html css import karleha hey

aur database connectivity environmrnts diyi hui hey

humko .env file b banani padthi hey , jaha sey database ko connect karnekey liye jo jo env chahiye vo idhar dey rakhey hey

directly pass kar sakthey hey yah env file mey b rakh sakthey hey

samaj mey aagaya na mynotes - fronend , notesapp -backend , nginx name ka folder jaha pey nginx conf rakhi hui hey

upstream django{ server django_app:8000; } server { listen 80; server_name localhost; location / { proxy_pass http://django_cont:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } }

ki agar is server pey port number 80 pey kahi pey http django container uska port 8000 url aathi hey vo / ko redirect kardega

agar mai instance ki ip:/ lagadun then ae redirct karega internal django product mey

clone hogaya - iski docker file dekhlenge

FROM python:3.9 WORKDIR /app/backend COPY requirements.txt /app/backend RUN apt-get update \ && apt-get upgrade -y \ && apt-get install -y gcc default-libmysqlclient-dev pkg-config \ && rm -rf /var/lib/apt/lists/* # Install app dependencies RUN pip install mysqlclient RUN pip install --no-cache-dir -r requirements.txt COPY . /app/backend EXPOSE 8000 #RUN python manage.py migrate #RUN python manage.py makemigrations

docker build -t noted-app .

mujhey three container banana hey -

vim docker-compose.yml

version: "3.8" services: nginx: build: context: ./nginx container_name: nginx ports: - "80:80" networks: - notes-app restart: always depends_on: - django db: name: mysql image: mysql ports: -p "3306:3306" environments: MYSQL_ROOT_PASSWORD: root MYSQL_DATABASE: test_db volumes: ./mysql-data:/var/lib/mysql networks: notes-app restart: always healthcheck: test: ["CMD", "mysqladmin","ping","-h","localhost","-uroot","-proot"] interval: 10s timeout: 5s retries: 5 start_period: 60s django: build: context: . container_name: django_cont command: sh -c " python manage.py migrate --no-input && gunicorn notes.wsgi --bind 0.0.0.0:8000" ports: -p "8000:8000" env_file: - ".env" restart: always depends_on: - db healthcheck: test: ["CMD-SHELL","curl -f http://localhost:8000/admin || exit 1"] interval: 10s timeout: 5s retries: 5 start_period: 60s volumes: mysql-data networks: notes-app

cd nginx/ cat Dockerfile , default file hey isko kaisey likhthey hey →go to google → nginx default config file - begginers guide - here you see the document

ab - dockerfile mey dekho

FROM nginx:1.23-alpine

COPY ./default.conf /etc/nginx/conf.d/default.conf

ab docker compose mey nginx ka likhdo , ab django mysql pey depend kartha hey toh pehle mysql likhdo

idhar django ka cont_name important kiunki cont_name humne default.conf mey mention ki hui hey

gunicorn ae aisa ek tool hotha hey jo aapkey aaplication ko server kartha hey production mey , toh gunicorn ek wsgi ek file rethi hey jo apney application ko server kartha hey

djangi k application hotha hey na , jab b vo mysql sey connect hoha hey na migrate karna padtha hey isliye command rehtha hey

ab docker compose up
ab sab issues fix kiya i hope it will work now - docker compose up —build -ae kiun pata hey by default ae cache ko letha hey isliye phir sey build karney key liye —build detheyhey ae like —no-cache

— ja thak mysql container dhang sey chalega thab thak django nahi chalega

jo health check mainey mysql key liye kiya vahi health check django key liye karun

docker compose up —build → docker ps -a

docker network create notes-app -d bridge

docker compose up —build

Three tier application with docker compose java app

→ GO TO Expense tracker → pehley ae project key bharemey dekhenge

ab samjo three tier application ka matlab kya hotah hey , ek hotha hey frontend -jo thyme leaf name ki framework hey - thymeleaf name ki ek frontend hey uspey aapka ui banahua hey

jo aapka bckend hey springboot hey ek java ka framework backend

jo data base hey mysql - pey data base hey

backend hey ae build hotha hey maven sey - maven ek build tool hey apache foundation banaya hey

apey ko docker compose chahiye ismey ae three containers banare honge

thymeleaf & springboot - ki ae do b ek application mey combine kar sakthey hey

jo maven key through build hojayega - base image rahega maven ka - phir maven ki base image mey soringbook java build karenge

aek mysql db lenge connect karneka khoshish karenge

ab thodasa dekhenge is project key bhareme →

src → developer ka kaam - yaha pey kuch test cases hey , main mey resources hey → assets - jo landing pages , css

application.properties ae important part hey

application ka name: expensesapp, spring.database: jdbc:mysql://mysql:3306/expenses_tracker , username= , password=

ae vo configurations hey ajo aapkey springboot app mey hey , springboot configuration hey vo kuch kuch variables aur uski corresponding values ko mangthi hey - vahi values mysql pey makesure karna hey ki aap derey ho

ab humko pom.xml important hey - aapkey application hey kounsey kounsey dependencies use kartha hey

Clone the code from shubham - > https://github.com/LondheShubham153/Expenses-Tracker-WebApp.git

pehley dockerfile and docker-compose.yml ko remove karo scratch sey banayenge

# stage 1 - Build the jar (Java application Runtime) FROM maven:3.8.3-openjdk-17 AS builder # Working directory WORKDIR /app COPY . . RUN mvn clean install -DskipTests=true #Stage 2 - execute the Jar File from the above stage FROM openjdk:17-alpine COPY --from=builder /app/target/*.jar /app/target/expenseapp.jar CMD ["java","-jar"."/app/target/expenseapp.jar"]

docker build -t expenseapp .

vim dokcer-compose.yml

```dockerfile version: "3.8"

services: java-app: build: context: . container_name: "expensesapp" networks:

expenses-app-nw ports:

"8080:8080"

environment: SPRING_DATASOURCE_USERNAME: root SPRING_DATASOURCE_URL: "jdbc:mysql://mysql:3306/expenses_tracker?allowPublicKeyRetrieval=true&useSSL=false" SPRING_DATASOURCE_PASSWORD: Test@123 depends_on:

mysql_db restart: always healthcheck: test ["CMD-SHELL" , "curl -f http://localhost:8080 || exit 1"] interval: 10s timeout: 5s retries: 5 start_period:60s

mysql_db: image: mysql container_name: "mysql" networks:

expenses-app-nw environment: MYSQL_ROOT_PASSWORD: Test@123 MYSQL_DATABASE: "expenses_tracker" restart: always ports:

"3306:3306" healthcheck: test ["CMD" , "mysqladmin","ping","-h","localhost","-uroot","-pTest@123"] interval: 10s timeout: 5s retries: 5 start_period:60s volumes:

./mysql-data:/var/lib/mysql nginx: build: context: ./nginx container_name: nginx_cont ports:

"80:80" networks:

expenses-app-nw restart: always depends_on:

java-appand

networks: expenses-app-nw:

```

- docker compose up

networks mey mapping agar deyaga toh - nahi rehna chahiye ; colon rehna hey

docker compose up —build

enable the port number in aws 8080

cp -r ../django-notes-app/nginx .

vim /nginx/default.conf -→ replace container name here and port number 8080

upstream server expensesapp 8080

copy nginx code from django naotes

and paste it here and docker compose up —build

why permission denied ari mysql-data ki → aapka jo volume hey created nahi hey

docker volume create java-app-data → ab apna docker compose.yml file mey replace kardo java-app-data

SSL B BATADO - take your ip address and go to go-daddy .com - create one profile and purchase one domain.

phir jakey us domain ka na dns mapping hotha hey -→ ab mai dns mapping mey ae ip address ispey ma kardo karkey -→ manage dns mey - aek record banado - add new record - docker name - option mey A record , Arecord subdomain hotha hey - > docker.trainwithshubham.com → value mey ip daldo aur ttl - custom seconds 600 and save it

ae domain name nginx → default.conf mey daldo → phir A record delete karkey phirsey create kardo aur

docker compose up —build and now check it will be working.

Docker scout

jabbi aap docker ki images banahey ho - lets say aap docker ki image banali - aap ney two-tier flask ki image banali

ae image aek bse image sey bani - ae python ki or kisi ki b ho sakthi hey right , aisa b ho saktha hey ae base image mey kuch security related threats , security related issues , vulnerabilities ho , isiliye image ko humne security key liye scan karthey hey

bhaiyya ae scan trivy b kartah hey , trivy ka kaam image , scanning , file system scanning k a

similarly usi prakar sey docker ney khud ka b image scanning tool banaya hey docker scout

aapko bohot aasan sey dockerdesktop mey hotha hey

desktop mey jakey - aapkey pass jithney b images hey docker scout scan kar saktha hey - dekh saktha hey ki kahi uski base image sey yah complete images ser kahi threat vulnerabilities hey kya karkey

ae issues kya hey kaisa pata chalega

terminal mey javo uskey liye - docker images

docker scout quickview imagename - ae pehley dockerhub sey authentic karega aur scan karkey diyega

aur depth mey janna hey - docker scout dves imagename

Docker init:

AE sa samajlo ki aapko dockerfile banani hey - dockecompose banai hey k readme banai hey kithna time lagtha hey

docker file mey kya kya likhtey hey

from , workdir , copy , run , cmd , author , label , maintainer

but ae jo kaam manually karna padtha hey

docker init kya kartha hey aapko ek ready made template bana detha ehy

mujhey just docker init karna hey - aur select karna hey aur click akrna hey

docker ignore bana detha hey , dockerfile , composefile , readme b bana detha hey - project select karlo

use version also - enter - relative directory of main landing package → .

port - 8080 ,

within few seconds - dockerfile , docker compose and sab files banayega .

Git and Github For DevOps

Vaseem Shaik — Wed, 28 May 2025 17:50:04 GMT

Agenda:

Git and GitHub Basics

Basic Commands of git

pull request, merging, branching strategies for DevOps

Hooks (Pre-commit Hooks)

Repo setup End to end & us repo ko (Github Actions key through CI/CD kaisa karenge)

Pehle samjengey kya hotha hey git and github

Git is a Version Control System - ae kya hotha hey - aap aisa samjo, ae aap ho, ae aapka dost hey, aur ek dost hey, mai aur babban achey vale dost hey - sochey sath mey boo likhenge - in book mey bohot chapters rehthey hey - mai chapter 1 karlunga, chapter 2 babban, chapter 3 gabban

phir mainey kithna kaam kiya, kithne time kiya, chapter 1 karnaa author kon tha, kya proof hey ae chapter mainey hi likhey aek commitment hothi hey —→ ae saari ceezey record hni chahiye agar aap group project kar raheho

Vara aisey hi log hothey hey pura assignment mila hey assignment mila hey, saara project mainey kiya lekin babban ney likha my contribution in the assignment.

Ae sab record honi chahiye - or let’s say mainey aapney book key chapter mey kuch jadbad kardi, main usko edit kar pau, or jabbi chahu purani version mey jau pau

aap versions pey ja sakthey ho, aap kisi aur key changes ko record kar sakthey ho - aap aek version control system mey kaam karreyho

Git b ek vcs hey jaha pey aap aur aapkey team mates ek repository mey aapkey contributions dal sakthey ho, aap apney code ko maintain kar sakthey hey

kithney bajey likha , kya proof hey , kya data hey , koun likha sab track kr saktha hey

Linus Torvalds has created git , how can we download the git , just install the git download software on windows.

git —version - to check the git version

Github kya hey : Github is a platform jiskey background mey git vcs ka fundamentals sari cheezey include hey , like apna code manage kar sakthey hey , collabration kar sakthey hey , llike gui dey detha hey .

github / gitlab kya difference - both are same = isko bolthey hey source code management tools .

Git is a vcs tool hey ae power kartha hey github, gitlab and bitbucket ko.

Go to GitHub.com and signup and create your account.

Basics Commands:

i will create a folder and file ismey mera code rahega - through git files ko manage karunga → phir github mey push karune valey hey , later on , pree hooks laganey valey hey aur github actions key through , pull request through , project skey through ci cd karenge

→folder→ filer→git→girhub→pre hooks→githubaction→pr→project→CI

let’s get started - i will create a new window in vs code i will create a fodler - click on open -go to document folder - create a folder githubfordevops→ open this folder

testing.py - print(“Heloo dosto , kya haal chaal , kaisey hey sab”)

ab is cheez ko , mera code hey , mainey likha hey , main is file ko run karunga

ab ae file kisney likhi , kab likhi , kya proof hey kisney likhi , agar ae file delete hogai kya ae file retrive hui

aapko git boltha hey mai aapko ek vcs detha hun ki jo is file likhne valae author nbata sakthe hey , data bata saktha hey , proof commit bata saktha hey , agar delete ho gayi kya mai v1 puraney version pey ja saktha hun no , git ja saktha hey

rm testing.py - kya ae retrive kar saktehy no

file system mey you cannot recover deleted stuff , in vcs you have retrive option and also roll back to the previous version

Mujhey abhi is cheez ko vcs banana hey is folder mey aapko karna padega git init

Git init : to initialize a empty git vcs repository local repo

aap is directory ko file sytem sey vcs ko git init

ek baar initialize hogai aapki git directory -

git status - to check the

create a file testing.py file - ab mujhey git status karkey dekhna

ab humko aayega untrakked files - hum jab local repo create karthey hey

ae 2 logical areas create kartha hey untracked , stage and tracked

ab jab b aapki file system mey hothi hey untracked hothi hey git vcs ko

agar aapko us file ko git vcs key andar dalna hey vo cheez tracked hojayegi

is file ko stage mey lekey aaunag aur track mey dalunga

ae ship mey jana hey yah nahi vaisa sochlo

untracked sey stage mey lana hey - git add file name

vapas untracked mey lana hey - git rm —cached file name

ab stage sey track karani hey - git commit -m “message”

git status - vo abhi tracked hey , agar mai testing.py ko delete kardunga - toh isko mai vapas restre kar saktha hun

git restore python.py

ab thak humney sikha - git init , git status , git add , git commit , git rm -cached , git restore

Ab nayi file banathey hey git-commands.txt - history commands ko copy karlo aur paster kardo

Hum abhi ae sare changes ko local sey remote github ko push karenge

Github pey jakey ek new repo create kardo - name - github-for-devops-workshop

aap ka jo local hey vo aapka personal hey , github hey world wide web pey hey - kahi sey b access kar sakthey hey , agar aapko aapkey local code hey vaha sey github ko pahunchani hey git push command sey paunchavogey

github mey username password lagtha hey na

mai use kartha hun , mai local ko personal access token de detah hun thaki vo merey github mey push kar detha

copy which are here push an existing repo - git remote add origin url

hey github , local ko batara hun remote dhoor mey rakhi hui ek url usko add kardo

git remote -v → remote add hui yah nahi idhar check karthey

git push origin main - jab enter karunga - username and token mangega

aap github settings pey jakey - personal access token -create karo - token ko repo ka access dedo

git remote set-url origin https://PAT@github.com/username/repo.git

git push origin master - push hogaya

Github mey rakhi hui repository , iskey sath ky akya kar sakthey hey , aap clone kar sakthey ho yah fork kar sakthey hey

fork - github - github hotha hey - fork karlo

clone - github to local clone hotha hey clone karlo

mainey commands jo b naye they remote add ka vo commands.md mey likha - git status kiya - git add kiya - git commit kar detha hun

bina token sey connect - settings mey jakey - ssh mey - apka system ka ssh-key copy karna hey thab vo allow karega

mai direct remote mey edit kartha hun - commit kardiya github key through

jo new sey change kiya kya vo merey local mey hey ? toh ek command use karunga - git pull origin master

Branches:

what is branches

-ae jo main master kya tha - ae branches key name hothey hey

branches kya hothey hey - tree pata hey , jaisey tree key roots rehthey hey - tree key seed lagathey hey pehley us seed sey rrrroots - roots sey - tree - trees - branches hothey hey

seed mathlab , jab git repo initally commit kiya create kiye na vo seed tha vo tha master pey - jab local mey repo banathey hey master pey rehtah hey

agar github mey create arangey na main ey rehtha hey

git branch branchname - create the branch

git checkout -b brnachname - create branch and also switch

git switch dev copy of the same desired state of code

dev mey jakey mainey create kiya ek file - add kardunga , commit kardunga

master ko paat nai dev mey kya chalra hey , dev ko pastha nahi master mey kya chalra hey

every branch in git maintains its own copy - faida - isolate , copy banalethey hey -

Branchin ki strategies rehthey hey

ek code rakha rehtha hey master pey

aur ek code rehtah hey stage - staging mey ky ahotha hey jo master ka code hey uat , users test kar paye

dev mey dev code rakha jatah hey sirf sur sirf development hothi hey

feauture1 , feautre2, feauture3 → dev → test (ua) → master

dev mey naya developer kuch enhancements key liye use karthey

agar aap chahthey ho ki kaha pey rakha hua hey production mey dalna hey yah nah dalna hey usey stagging mey rakha jatha hey

aur sahi vala hey , sahi sey chalra hey usko checihi math isliye vo code - master pey rehthi hey

bohot log kaam karthey right feauture pey so ek ek person vo khud branches create karlega feauture key uppar

mujhey dev key changes master mey merge karna hey toh make sure your on master - and then git merge dev

git hub key though compare and save

#if you go git tech fetch , jithney b branches remote me hey local mey aajathey

GIT HOOKS:

Yeh aapki branch hey master , ae daigram dekkey kya yadh aatha hey - ki untraced , stage aur tracked yadh aara , aisa samajlo ki mainey ek untracked file thi

lets say demo.py - usko mainey stage karnekey liye git add kiya , bohot saare files thi git add . kiya git commit kiya , kya vo track hojayegi

mujhko ae batao , demo.file mey mujhey galthi api-key reh gayi , kahi code mey kuch galath tha kya vo commit hogayi

galthi sey galath code b ho saktha hey , it is humar error

iska solution kya hotah hey - commit karnesey pehle badh , push karnesey pehley yah badh - kuch internal checks laga sakthey ho

aur ae jo add aur commit key beechmey aek check ho saktha hey - ae automatically check hona hey toh hooks sey hoga - pre commit hook ser agar aapkey files mey dikath hey , aur aap commit karney jathey ho - syntax wrong hey toh ae commit nahi kar detha

ab mai apney rep mey jaunga , ls -a , aur .git valey folder key andar jav - cd .git - gaye bohot dhyaan sey dekhna - ls kiya idhar hooks name ka folder dikhega

ls - pre commit , prepare-commit dikhege - bohot saare samples hey , apko actually ae name ki file banana hey →

```plaintext vim pre-commit

vs code mey jakey - search mey user.exclude - idhar .git lo delete karni hey kiun ki mujhey visible hojaye

mainey ek file banayi jiska name demo.py - jiskey andar ek mainey function banaliya

demo.py

def my_function(): a=5 b=6 c=6 test() return d my_function()

ae wrong code hey agar mai git status karun , git add and commit kardun , ideally

commit hojayega -> git ko thodi hi patah hey ae wrong code karkey

this file is now addes , thabki ae file galath file hey

mai isko pre commit hooks sey theek kar detha hun

toh python ka jo code rehtah hey usko check karnekey liye -> flake8 hotha hey code ka quality check kar saktheyy hey

pip install flake8

flake8 demo.y

is tool sey code ka quality check karliya , ae code quality test kartha hey

kya mai as a devops engineer , ae tool ko apney git key process mey daldun

commit honesey pehley ae tool chal jaye

mai kya chahtha hun , hooks mey jaunga ek file banaunga - pre-commit file

ab mai aapni sarey files ko lekey aajaunga

files=$(diff --cached --name-only --diff-filter=ACM | grep '.py$' flake8 $files

if git grep - q "password|secret_key|API_KEY' $(git diff --cached --name-only); then echo " Kiun apni job risk" exit 1 fi

ab mai pre-commit file ko executable banadunga

chmod u+x pre-commit

ab mai git status , git add . - git commit -m "commit" galath file mai commit

karney try karra hun lekin pre-commit hook commit nahi karney dega

\- ky aap aapney office mey puri ki puri pre commits hits lagagkey purei ki puri teams ki jo bugs , they problems they ek hi minute mey solve kar sakthey , jo wrong code dalenge hi nahi toh problems ayeney hi nahi * agar mai galthi sey password , yah api-keys hey toh mai commit nahi karney dena . * mai demo.py mey password daldiya , ab git add . and git commit khoshish karunga toh error aajayega # Github Actions: * Github actions key sath aek portfolio deploy karna chahthey ho \- ab aisa samajlo ae aapki github repo hey , jiskey andar aapkey files hey , ab ismey ek branche hey master , uskey andar kuch kuch commands hey application hey is application ko deploy karthey ho ,isko aap build ,test , phir deploy b karthey ho right , is cheez karnekey liye jenkins use karthey hey right ci cd pipelines banathey ho right - > ab ae jo pipelines hey jo jenkins mey banri vo aap github mey b bana sakthey ho , jisko bolthey hey github actions , github actions aek inbuilt tool hey jo aapko aapkey code ko build test deploy , scan , sonarqube , sab cheez karney detha hey * `aapko jana hey aapkey repo mey jana hey - lets say isko scan karna hey - actions mey javo - pylint - configure - khud hi likh diya python code test key liye pylint - yaml mey rehtha hey` aapkey repo mey .github folder rehtah hey usmey aap workflows likh sakhey hey ki - onpush , ontest, ondeploy commit karney key badh- github workflows key bajuk small dot rehtah hey - vo actions run horey hongey testing.py mey thoda code likhdo / def heloo(): “““ this is ehloo function “““ return “heloo dosto” heloo() tum commit karkey push karenge automatically aapkey actions start honge run karna * jo b error aaya hey as a python epert karkey aapka file ka code and error dekey fix akrdo karkey batao aajayega aur copy karkey add and commit push kardo. Go to aws account : - s3 → mai ek bucket banaunga - tws-portfolio - url aisa aek portfolio banaunga —> through github actions Github pey aapka code rakha jayega code hoga html , java script aur css , java script → is code ko s3 bucket mey oaunchana hey - is bucket ko public access dekhey - public thak paunchana hey - ae sab ho automated github actions s3 ko access karnekey liye - iam - security creds hothi hey , create kardo acess key and security key * tws-portfolio fork karkey → uskey andar html css aur java script ka code hye * mai ek file create karunga .github/workflows/main.yml * * name: portfolio ```plaintext name: portfolio on push branches - main jobs: build-and-deploy runs-on-ubuntu-latest steps: - NAME: checkout uses: actions/checkout@v1 - name: configure aws cred uses: aws-actions/configure-aws-credential@v1 # humko is repo key settings mey secrets and variables mey actions mey secret and access key banado with: aws-access-key-id: ${{ secrets.aws_access_key_id}} -
-s3 -mey jakey cukedt create karkey - s3 hosting mey index .html create karo

aur bucket policy mey jakey - gnerator policy - s3 bucket policy mey allow - bucket policy copy kareleye - bucket policy create kardo replacing with your bucket id

Linux End To End

Vaseem Shaik — Tue, 27 May 2025 19:29:31 GMT

How Internet works

Internet works through optical fibers cables indise the water they keep cables and attach with the data centers

data transfer through cables , this cables are located in the sea.

between user and cables -data center, a ISP will be there ,

what is server :

a server hey uska kaam kya hey -kisi info ko server karna , server is just basically a computer to server client

the server is to server the info

client ka kaam hey request info to the server

youtube.com ae aek domain hey , iskey against ek application server i mean application ka domain

ek application server hoga jaha pey youtube chalra -is server ka kuch ip address hogi → aek domain aur ip address ko link karnekey liye ek DNS server rehtha hey

facebook.com kultha hey - isp internet ka acess detha hey , oprical fiber data ko flow karathi hey - aur data center mey jo application chalra co aapkey mobile mey stream hotha rehtha hey

diff between web server and application server

web server is nothing but jo b static code deploy kiye rehthey toh vo web servers , static data serve kartha hey

application server bussiness logic data ko server kartha hey

Types of applications

standalone apps - usko internet ki jaroorat nahi hothi —> koi db ki jaroorat nahi rehthi - aap kabhi airport mey , how was your feedback , click kar diya chaldiya - usko bakhi cheez ka jaroorat nahi hey

if you have web application like instagram or youtube , youtube mey video streaming hora hey , Notifications jarey ae thab hotha hey aap standalone nahi aap ek group of applications chalare hey - iskey supporting mey app severs, db servers, email server

isiliye ek devops engineer ko pata rehna hey ki vo kis cheez pey kaam karra

agar stahndalone pey kaam karra hey usko db , ki bakhi connections ki jaruru nahi padegi

agar ae web applications pey kaam karra hey isko db connction , frontend connection , backend connection , cloud ae sab dekhna

what is application support and maintainance

ki jo hamare phone mey laptop mey application horey hey ae ssaarey applications kahi ek o.s mey chalre hothey hey

vo applications sahi sey cha;re hey nahi , creash toh nahi horey , us appliocations ko support ki jaruri hey yah nahi dekhthey hey

application chalra hey kabhi bhi connection issue aa ra hogey as a devops or linux admin kounsey server mey dikkath hey

ae applications linux pey chalthey hey , linux ae ky ahey cheez

90 percent key log windows chalathey hey 90 percent applications linux mey chalthey hey

linux opensource hey , linux ko linus torvalds ney 1991 mey banaya tha , jissey jo basically koi antivirus ki jaruri nahi hey , security top notch hey , community hey , multi tasking hey , updates ki jaruri nahi ,

linux use karthey hey majority companies linux pey hi chalrehey

ae jo unix and linux kya hey - unix basically ek o.s hey vo ek commercialised hey - unix ka prime example hey mac o.s

same as liunux kaam sab linux hi hey .

unix ka hi free version linux

\linux ko install kaisa karey - pehle tarika hey

1. wsl for linux

virtual box

azure / aws/gcp koi b cloud pey koi vm banakey use kar sakthey hey

vagrant b hey - hashicorp ki company hey ussey aao linux use kar sakthey hey

Difference between linux and windows

windows is a licence one

linux is a gpl general public license

linux mainly focus on development , networking , tyerminal , programming , scripting \

windows mey antivirus use karna , linux mey nahi

widnows mey update pusthey hey , linu smey update ki jaruri nahi - ubuntu vala saal mey 2 baar update martha hey

softeare remote location server tools

agar ae mey hu aur yaha mey mai ek dhoor rakhey huey server or vm ko connect karna chahtha hun

remote desktop protocol

2. ssh - secure shell

3. anydesk

4. ultraviewer

what is kernal , bootloader , shell

ki linux kya hotha hey , linux ek o.s hotha hey - har ek o.s ka dil rehtha hey - ud dil ko bolthey hey kernal ]

kernal basically purey o.s ko chalaney liye jo coding files , processors , program required hothi hey vo kernal key pass hothey hey

kernal ka founder linux - ab is kernal ka kuch kaam hotha hey processes chalana ae hotha hey

ae c program mey likha hua kernal hey , humko c nahi aathi , hum kaisa chalaye , hum ko aek program chahiye is kernal ko bath karey thab aajatha hey shell

shell basically ek programm terminal hotah hhey jaha sey kernal ko bath kar pathey

* shell key commands rehthey hey , un commands ko use karkey shell commands sey hum bath karthey hey

* shell ek gateway hey aapko aur kernal key beech mey rehtha hey

* bootloader ky ahotha hey - aisa samajlo aap ka computer start hotha hey

boot loader ek aesi proceesor hey jo o.s mey files rehthi hey kerma; mey rakhey huey process ko jagah detha hey - gnu - grub - - it helps to boot your entire system

* grand unified boot loader (grub)

aapkey center mey rehtah hey kernal - kernal key uppar shell - shell key uppar application utilities rehthey hey

shell kernal sey kernal - hardware sey bath kartha hey = kernal change karega hadrware ko

df -h - command jo b disk ka infro hey dikh jayega

free -h = system memory usage ., ram usage check kartha hey

cpu ka usage - Top command sey dekh sakthey hey

Linux file system kya hotah hey

in lunux everything in system treated as files , everything starts from / - isko bolthey hey / root folder - iskey andar bohot saare folders rehthey hey

cd - change directory - ek directiory sey dueri directoey ko jana hey toh cd use kartha hun

cd / - ismey aapko bohot saarey folders dikhthey hey

linux harrek cheez ko segregate kartha hey - agar merey logs aaaye toh linux var mey store kartha hey

agar jithey mey b linux users hey - usr mey store kartha hey

toh linux mey harrek cheez file system sey based hey

#state of proceeses in linux : agar aapko cpu mey kya chalra hey ytoh top - aap dekh sakthey ehy kithey processes het , kithne chalre hey kithey sokey hey dekh sakthey hey

actually ae background mey process id sey run hothey hey

linux commands ":

date : date is used to see teh current time and date

ls - list of files it shows cin current directory

mkdir devops - it creates the directory

ls -l - its hows the detaills of file or folders - permissions , owner of thr filr or dir , sixe, date and time and file/folder name - first character denotes teh file or dir

pwd - is used to see hwere we are currently

touch - is used to create a new file

clear - it cl;ears the terminal

cd - change directory - cuurent directory sey kisi aur directory ko jana hey

cd .. - one directory back

rmdir - deletes the empty directory

rm - delete the files

rm -rf folder - if folder has any files or subdirectories - recursively deleted

cat - file ko dekhna hey toh cat use kartha hey

agar file mey kuch print karna hey “ echo “ hello world” > demofile.txt

redirect karne | echo "“how are u “ » demofile.txt

zcat : agar aapki koi zip file hey compressed file hey vo compressed file ko dekhna hey contents ko dekhney zcat use karthey

head : first 10 lines dekhna hey file key head use karthey hey

tail - nichey key 10 liens dekhne use karthey hey

tail -n number filename

more and less aisa samaj lo aapkey pass badi file hey , agar file ka content ek page ,dusa page , i mean page by page screen dikhayega

it shows in pagenated way

cp : mujey ek file or folder ko kahi aur pey copy karna hey mujhey cp command use karna

cp oldfilename newfile name - backupfile hojatha

cp -r cloud/ devops/ - cloud folder ko devops name key folder mey copy kar diya

mv : mujhey copy nahi move karni hey file i will use mv command

rename karni hey thabbi mv use karthey hey

mv oldname newname

wc: agar muhey ae file.txt mey kithey words , lines , characters / kithni storage ki hey bytes hey toh mai wc command use kar saktha hun

hardlink : dekho jab windows mey aapko game khelna hotah hey c mey rehtha tha aap hamesh thodi hi c mey jakey khelthey usko shortcut banathey hey dekstop mey

links are shortcuts - hardllink ae aisa shortcut rehtha hey jaha pey agar aapka jo main vali file delete hogi bhi shortcut hey zinda rehtha hey

softlink aisa hey agar main file dleet hogayi shortcut b delete hojathi

echo “hi “ > devops.txt

* ln /home/ubuntu/cloud?devops/devops.txt hardlink-file

* ln -s /home/ubuntu/cloud/devops/devops.txt softlinkfile

* ls -ltr , toh agar orinal location mey content change hua toh softlink mey bhi change hojatha

cut = cut kya kartha hey jo file mey content hey like “ this is my file” - aisey 4 words hey lets say ek word 2 bytes ka hey

cut -b 1 myfile.txt → cut -b 1-4 myfile.txt

file mey sey chotasa portion chahiye cut karkey dey detha

tee - echo “ heloo” | tee - toh vo cheex screen pey b dikhayegi aur aapko file mey dalni hey vo file mey b dal dethi hey

echo “heloo” | tee heloo.txt

to take input and display the output on screen and also in file i have to send

sort : it allows to sort the order in alphabbetically

clear - clar the terminal

diff - mujhe do files mey kya diff hey dekhna hey

diff demo.file hardlink.file

vim : vim ek editor jaisa hey edit karne key liye

ssh : merey pass ae linux instance hey ae kahi dhoor pey rakha hua instance hey agar muhey udhar thak paunchna hey through ssh sey

remote server ko secure shell sey connect karna hey toh ssh use kar sakthey hey ssh ko kuch ports use karthey hey 22 ae enable rehna hey

aap rahul aap anjali sey bath karreho , ab anjali aapsey thabhi bath karegi anjali ka numer aapkey pass hoga - anjalaki ka number private hey - anjali rahulko degi , rahul public name hey na toh rahul anjali ko pata hona like public key agar anjali key pass public key hoga toh ki connect hoga

ssh -i pemfile usernameipaddress

aur putty b use kar sakthey hey , mobaxterm use kar sakthey hey

df : tumara system hey usmey kithna storage use hua kithna bacha hua hey dekhme key liye dekh sakthey hey

ismey itney blocks hey ithney used hey ithney available hey

storage key bharame dikhatha hey

du : disk utility - kisi particular folder or file ka sixe kithna use kiye dekhnekey liye use karthey hey

ps: processor ps batata hey ki particular command bash hey vo counsi processor pey chalra

top : total saare processor live mey batata hey

fuser : fuser vo ek particular file key liye ,

kill - kisi process ko kill karna hey kill use karthey hey

free : merey pass ithna ram ithna used , ithna available hey regarding ram dikhayega

nohup: generally applications mey use karthey hey - aek application mey loga aarey hey us loga ko kisi file mey store karna hey

nohub free -h - ek file banadega nohup,out usmey iska output daldega

agar mai chala unga nohub df -h

pehla vala output aisa hi rahega aur dusra vala b aatha hi collect hotha hi hoga

vmstat - jo virtual memory rehthi hey aapki vo dikhadetha

vmstat -a virtual memory mey ithna free hey ithna active inactive ram key bharame bata detha hey

system level commands :

jabbi ek devops engineeer apna sysem open kartha hey usko system key bhaaremey jankari reha hey , kounlogged in kiya kkounsey user sey logged in hey , kounsa o.s hey kithne der sey chalra , last logged in kab hey

uname: aap kounsey platform mey linux chalare ho jaisey ubuntu - mujhey pata chaltah hey kounsa platform chalara hun pata chaltha hey

uptimr: aapka system kithni der sey chalra hey - last 10:41 up 9 min sey , load average - 1 user active hey

date : current date na dtime

who , whoami key dif

who - kounsey user ney loginkab login kiya tha , kithney users ae system ko logged in kiye hey

whoami - system meu current user logged in user hey uska user name dekhna hey whoami

agar uskey sab activities dekhna hey who likhdunga

which : ae devops ko - let say merey pass bash installed hey kounsa bash installed hey

which bash - vo batadega us batch ki location

which cp : insatlled ka location batadega

id : ae bada hi important cheez hotha hey - jis user sey logged in hua uska id kya hey , uska group id kya hey

id - ubuntu ka id , ubuntu gid ae sab details batadega

sudo - aap aur ek user banaya same gharoe rehre aap kithna package kamathey hey ae sab farak nahi padtha gharpe sudo papa root ki chalegi

agar aapka ghar linux ka machine hey jithna b users add chalalo papa si toh koi bada nahi rehtha root user key pass saari permissions rehthi hey

super user - ubuntu jethalal ki file delete karna hey iskey pass permissions nahi rehtha thab sudo ka permission letha hey

agar aapko kisi cheez ki permissions nahi mil pari toh sudo use karlethey

cat /etc/groups

mujhey agar sudo use karunga yag sudo group mey add karunga

shutdown: aapkey systemko shutdown karni hey papa ki permission lekey shutdow karo

reboot: system ko restart karne key liye use karthey hey

apt , yum , dnf , pacman

apt - application package manger ubuntu package install karnekey liye use karthey hey

sudo apt install docker - docker ko isi system mey dhundra hey , heyi nahi toh internet sey lekey aaunag

sudo apt-get docker.io - internet sey lekey aakey download karrahun

sudo apt remove docker.io - uninstall hojayega

YUM , DNF , PACMAN - AE b package managers hey alag alag distributions ka

centos- yum , dnf - fedora ka , pacman - arch , redhat - rpm

which docker - kaha pey install hua hey dikhayega

users and files :

linux famous hua because multi user feauture hey

user kaisa create karthey hey - useradd -m ajay - permission denied - without sudo you cannot create

sudo sueradd -m vaseem - creates user wih home directory

passwd vaseem - creates the password

su vaseem - switch user - agar merko user switch karni hey su use kar saktha hun

whoami

pwd

userdel : user ko delete karna hey - > sudo userdel ajay

grouppadd : ap as a devops engineer ek company mey add hothey hey , humdevops engineer hey , bohot saare users add huey company mey , 3 log testers , 2 log devops key

in devops logon key roles and permissions honge , as a devops engineer in sab ko kaisa add karenge aur roles and permissions denge

toh mai group banaunag tester and devops us group mey test employee ko tester group mey add , aur devops valaey ko devops group mey

#sudo useradd 1, 2, 3,4

#groupadd devops test

#cat /etc/passwd

#cat /etc/group

gpasswd: group mey members ko add karna hey

sudo gpasswd -a ajay devops

sudo gpasswd -a devops devops

sudo passwd -a rohit tester

sudo gpasswd -M kohli,rahul,dhawan tetser

groupdel : sudo groupdel tester ; agar group delete karni hey th ae command use karthey hey

file permission commands

ae groups and users ko kiun create kiya - ae isloye kiyatha un users ko in files and folder ks paoper access miley

file ko permission ki kya zaroorath hey

agar ls -l kiun toh iska info aaga

first charcter - defines either file , folder or link than next permissions - user , group other

r= read =4 ; write =2 =w ; x=execute=1

chmod : agar merko file ka permission change karni hey mai chnagemod - chmod use kartah hun

chmod 777 filename / folder name

umask : umask 0002 - ki jabbi koi file create karthey ho by default permisions kya honi chhaiye iskey liye umask rehtha hey

chown : used to change owner

chgrp used to chnage group

compression command :

zip , gunzip ,and gzip : agar merey pass bohot saarey files anf folders hey toh isko mai zip compress karna hey ae commands use karunag

zip -r backup.zip folder

unzip backup.zip

1.1 : tar : tar -cvf file.tar cloud/

tar , untar

scp : secure copy agar merko local sey remote , remote sey local ko files copy karni hey toh scp use karthey hey

scp -i pem filename usernameipaddress.

scp -i pem username@ip:remotefile loacl path

rsync : same as scp but here it works as a remote sync

rsync -e “ssh -i pemfile” -avz linux.file ubuntu@domainorip:/home/ubuntu/

Networking commands:

aap ho ek devops engineer - day to day life mey kartha kya hey - unka kaam zyada applications systems pey deploy karna

pehle devops kya karega agar application kaam nahi karra - us application ko ping karega

ping : ping ek command rehthi hey jo aapko batathi hey ae particular cheez po data recieve hori hey , data pahunchpara yah nahi

ping sigmasoftinfotech.com — main pahunch para yah nahi yah mere thak data pahunchra hey yah nahi

aapka internet b chalra aur revice b hora

ping internet bhejtha hey aur recieve b kartha hey

netstat - netstat ae rehtha hey matlab network statistics - network key related jithney b cheeze b hey bvo batata hey

bohot imp command - netstat ney kya kiya - top pey active internet connections - matlab aapkey system pey active internet connections ko dikha detha hey

kissi server pey active connections and protocals kounsey kounsey hey

jab tum ec2 use karthey hey bohot saarey sockets use hothey hey , sockerts matlab aisa samajloki one system sey dusri system sey yah type key interface sey connect karna hey sockets use hothey hey

net-tools key andar - netstat ,,traceroot , telnet , ns-lookup ae sab particular tool kit key andar aatha hey isko use karnekey liye net tools install karni hey

ifconfig : bhaiyay ifconfig kya hey ; interface

dockero , eth0 , i0 aatha hey - jithney b network interfaces hey idhar dikhata hey

ethernet cable eth0 - docker ek interface chalatha hey jab install karneke badh , ae ethernet kis cheez key connect hua rehtha (NIC) network interface card ae network ko jodneykey liye kaam aatha hey

l0 matlab loop back - matlab aapka jo system hey - lets sey nic kuch b connect nahi hua - local server ko loop back server bolthey internally chalega hamare server mey

traceroute kya hey tracepath kya hey

ae duniya mey hamko aek jhoot batadiya - server kaisa work hotha hey ae same concept - is cheez ko bolthey hey route is route ko trace karnekey liye trace route command use karthey hey

traceroute youtube.com-

sabse pehle traceroute kya kartha hey - pehle kaha gaya phi ekounsey ip pey gaya trace karthey hey kaha kaha pey hops huey dekhney key liye

ipaddress finder open karkey check karo

tracepath sigmasoftinfotech.com

sabsey pehle local host - sey kaha kaha hops hora karkey

mtr = ping+tracepath ping b karega , path b batayega

mtr sigmasoftinfotech.com , idhar 9 baar hops kartha hey , traceroute mey 30 baar route hos kartah hey

nslookup : nslookup sigmasoft.com - koi domain dhang sey chalra active hey yah nahi aur uska ip address '- active ey yah nahi dikhatha hey

telnet : telnet b same kartah hey particular website domain and port per connect karnemey help kartha hey

hostname: batadega ki is system ka hostaname kya hey

agar merko hostaname ka name chane karna hey sudo hostname sigamblr.com

cat /etc/hosts - idhar b chane kar sakthey hey

ip : apne computer ka adress dikhana hey - ip address show

iwconfig - jithney b wireless connections hothey hey — mujhey ab thak - ip , if , iw mila

aapko wireless cheeze karni hey wireless-tools install karna padtha hey

iwconfig - agar koi wifi -wireless hoga toh dikhadega

ss - ky hotha hey jaisa netstat hey na same ss detha hey

dig : agar aapko kisi b domainname ka info nikalna hey

kaha pey host kiya hey

whois - kaha sey kharida hey domainanme - kaha sey registry hui hey toh whois use karo

nc:

arp: address resolution , aapka device ka ddress rehtahey host ek router sey mac address mangatha hey vo router mac affress detha hey vo janneyka command arp.

ifplugstatus: batata hey aapkey interfacces chalre hey yah nahi

ae comands jo b ab padthey hey ae use karthey hi karthey hey

curl vs wget : curl jo hotha hey na , ae jo curl ae command aapki end points ko call karne key liye

api kya hotah hey ae aap ho aur vo aapka server - aap directly sevrer sey bath nahi karthey usi tarikhey sey ae aapho aur ae aek restaurant hey mcdonals aap directly kichen pey nahi ghusthey - waiter ko bolthey hey waiter na api hey

user will ask api to get something from server then server give the respone .

curl -X GET apiendpoint | jq - agar tum ko is reponse ka data ko sundar sey dikhana hey jq likhdo

wget : aisa samajlo internet se kisi cheez ko download karni hey wget use karthey hey

wget url

iptables : pura ek jaisey , vpc peering mey route tables rehthey hey - ip : rote table sab dikhatha hye

sudo iptables —list-rules

watch - jo command hey ae agar aapko koi b chalani hey uski output har ek 22 seconds pey dekhni baar baar dekhni hey

watch -n 5 top , watch top

nmap : bacially network map rehtha hey - pura network map - kind of route tables - ae ip kidar jatha hey , kya kya port sopen mey hey dekh saktheyhey

nmap -v sigmasoftinfotech.com

route : agar aapko aapne routes change karni hey - gatewya dikhatha hey - route table , inetrenet kaha sey route hori hey , subnets kaha sey route horey dikhatha heyy

pro linux command s:

awk

sed

grep

shellscript nahi likhni commands se hi khelna hey

toh awk hey ek command hey khub hi programming hey awk command : aek file hey us file mey , kuch range sof line slikalni hey , count dikahan hey kuch log files hey us log files mey key kuch data extract karna hey ae saari cheezon ko kaisa help karthi hey

download one log file ,

head -n 2 app.log

mujhey data time chahiye

awk ‘ {print}‘ app.log -→ awk ‘ {print $1,$2}’ app.log -→

mujhey sirf info lines chahiye -→ awk ‘/INFO/ {print $1,$2,$3,$4}’ app.log > info.log

kithni baar info aaya hey -→ awk ‘/INFO/ {count++} END {print count}’ app.log

awk ‘/ipaddress/ {count++} END {print count}’ app.log

ae logs hey 8:40 pm start huey aur 8:53 mey khatam huey mujhye 8:53 key hi logs chahiye

ae time kounsey column pey aaya hey

aws ‘ $2 >= “08:53:00” && $2 <= “08:53:59” {print $2,$3,$4}’ app.log

awk ‘NR >==2 && NR <=10 {print NR}‘ app.log

awk mey jo data chahiye na aek format data chahiye , , formated data , csv , tsv - format data hey awk chaltha hey

SED: sed hi awk jaisa tool hey stream editor - jo b aapka streaming data aara - vo real time mey edit b kar sakthey hey

sed -n ‘ /INFO/p‘ app.log # yaha apko expressions dena padthi hey exact match karneke liye -n dena padtha hey , pattern match ko p dalna hey

sed , line by line kaam kartha hey , jaha jaha pey i ibfo likha mujhey INFO hey usko replace kardo LOGS sey jo is cheez ko karthey globally kardo

sed ‘s/INFO/LOG/g ‘ app.log - info kounsi line pery aay ahey

sed '-n -e ‘s/INFO/'=’ app.log → = showing line number , row also print karani hey → ed '-n -e ‘/INFO/'=’ -e ‘/INFO/p’ app.log

-sed ‘1,15 s/INFO/LOG/g;1,10p; 11q ’ app.log #; this symbol is pause the command

Linux :

1. How the Internet Works

The internet is a global network of computers that communicate with each other to share information. Let’s break it down step by step:

Optical Fiber Cables:

The internet uses optical fiber cables, which are thin cables made of glass or plastic that carry data as light signals.

These cables are laid under the sea (submarine cables) to connect continents and countries. They transfer data at very high speeds.

Example: If you’re in India and accessing a website in the USA, your data travels through these undersea cables.

Data Centers:

Data centers are large buildings filled with powerful computers called servers. These servers store and process data for websites, apps, and services like YouTube or Facebook.

The optical fiber cables connect to these data centers to send and receive data.

ISP (Internet Service Provider):

The ISP is like a middleman that connects your device (phone, laptop) to the internet.

Example: Companies like Jio, Airtel, or Comcast provide internet access to your home. They connect your device to the optical fiber cables and data centers.

How Data Flows:

When you open YouTube on your phone, your request goes through your ISP, travels via optical fiber cables to a data center, and the server in the data center sends the video back to you.

This happens very fast, so you see the video streaming on your phone almost instantly.

2. What is a Server?

A server is just a powerful computer that provides (or “serves”) information or services to other devices, called clients.

Server’s Job:

A server’s main job is to respond to requests from clients. For example, when you visit a website, your browser (the client) asks the server for the website’s data, and the server sends it back.

Example: You type “youtube.com” in your browser. The YouTube server sends the video or webpage to your device.

Client’s Job:

A client is your device (phone, laptop, etc.) that requests information from a server.

Example: Your phone (client) asks the YouTube server for a video, and the server delivers it.

Domain and DNS:

A domain is a human-readable name like “youtube.com” or “facebook.com” that points to a server.

Every server has an IP address (like 192.168.1.1), which is like its unique address on the internet.

A DNS server (Domain Name System) connects the domain name (youtube.com) to the correct IP address of the server.

Example: When you type “youtube.com,” the DNS server finds the IP address of YouTube’s server and connects you to it.

3. Difference Between Web Server and Application Server

Web Server:

A web server delivers static content, like HTML pages, images, or CSS files, to your browser.

Example: If you visit a simple webpage with text and images, the web server sends those files to your browser.

Common web servers: Apache, Nginx.

Application Server:

An application server handles dynamic content and business logic. This means it processes data, runs calculations, or interacts with databases to create content on the fly.

Example: When you log in to Instagram, the application server checks your password, fetches your posts, and sends them to your phone.

Common application servers: Tomcat, JBoss.

Key Difference:

Web servers are for static content (pre-made files like images or HTML).

Application servers handle dynamic tasks (like processing login requests or generating personalized content).

4. Types of Applications

Applications can be divided into two main types:

Standalone Applications:

These apps don’t need the internet or a database to work.

Example: A feedback app at an airport where you click “Good” or “Bad” and it works offline without needing a server or database.

They are self-contained and run on your device alone.

Web Applications:

These apps need the internet and often rely on multiple servers (application server, database server, email server, etc.).

Example: YouTube or Instagram. These apps stream videos, send notifications, and store data in databases, requiring a group of servers to work together.

They need a frontend (what you see), backend (business logic), and database (to store data like posts or videos).

5. Application Support and Maintenance

What It Means:

Applications (like apps on your phone or software on your laptop) run on an operating system (like Linux or Windows).

Support and maintenance mean ensuring these applications run smoothly without crashing and fixing issues if they arise.

DevOps or Linux Admin’s Role:

A DevOps engineer or Linux admin checks if the application is running properly, if there are connection issues, or if a server is causing problems.

Example: If YouTube stops streaming, they check which server (web server, application server, or database) is having an issue.

6. What is Linux?

Linux Basics:

Linux is an open-source operating system, meaning anyone can use or modify it for free.

It was created by Linus Torvalds in 1991.

Most servers (90%) run on Linux because it’s secure, doesn’t need antivirus, supports multitasking, and gets community updates.

Why Companies Use Linux:

It’s free, secure, and highly customizable.

Most web applications (like YouTube, Instagram) run on Linux servers.

Linux vs. Unix:

Unix is a commercial operating system (like macOS).

Linux is a free, open-source version inspired by Unix.

Example: macOS is Unix-based, while Ubuntu is a Linux distribution.

How to Install Linux:

WSL (Windows Subsystem for Linux): Run Linux inside Windows without a separate machine.

VirtualBox: Install Linux on a virtual machine on your computer.

Cloud: Use cloud platforms like AWS, Azure, or GCP to create a Linux virtual machine.

Vagrant: A tool by HashiCorp to easily set up Linux environments.

7. Linux vs. Windows

Windows:

Requires a license (you pay for it).

Needs antivirus software.

Regular updates are mandatory.

Focused on user-friendly interfaces for general use (like gaming or office work).

Linux:

Free under the General Public License (GPL).

No need for antivirus due to strong security.

Updates are optional (e.g., Ubuntu releases updates twice a year).

Focused on development, networking, programming, and scripting.

8. Remote Server Connection Tools

To connect to a remote server (a computer far away), you can use these tools:

RDP (Remote Desktop Protocol): Connect to a server’s desktop remotely, like seeing its screen.

SSH (Secure Shell): Connect to a server’s command line securely using a terminal. Uses port 22.

AnyDesk: A remote desktop tool for graphical access.

UltraViewer: Similar to AnyDesk, for remote desktop access.

9. Kernel, Bootloader, and Shell

Kernel:

The kernel is the core of the operating system. It’s like the heart of Linux.

It manages hardware, processes, and files. It’s written in the C programming language.

Example: The kernel controls how your CPU runs programs or how your hard drive stores data.

Shell:

A shell is a program that lets you talk to the kernel using commands.

It’s like a bridge between you and the kernel.

Example: When you type a command like ls, the shell sends it to the kernel, which lists the files.

Bootloader:

The bootloader is a program that starts your operating system when you turn on your computer.

Example: GRUB (Grand Unified Bootloader) loads Linux by finding and starting the kernel and other necessary files.

How They Work Together:

The bootloader starts the kernel.

The kernel runs the operating system and hardware.

The shell lets you interact with the kernel by typing commands.

Example: You type df -h in the shell to check disk space, the shell tells the kernel, and the kernel gets the data from the hardware.

10. Linux File System

In Linux, everything is treated as a file (even folders, devices, or processes).

The file system starts with the root folder, written as /.

Key Directories:

/: The root folder, the top of the file system.

/var: Stores logs (e.g., system or application logs).

/usr: Stores user-related data and programs.

/home: Stores user files (like your documents or downloads).

Navigating the File System:

Use cd to change directories (e.g., cd /var to go to the var folder).

Use cd .. to go back one directory.

Use pwd to see your current directory.

11. Linux Commands

Here’s a breakdown of the Linux commands you mentioned, explained simply:

date: Shows the current date and time.

Example: date → Output: Tue May 27 03:00:00 IST 2025

ls: Lists files and folders in the current directory.

Example: ls → Output: file1.txt folder1

ls -l: Lists files/folders with details (permissions, owner, size, date).

Example: ls -l → Output: -rw-r--r-- 1 user group 1024 May 27 file1.txt

mkdir devops: Creates a folder named “devops”.

Example: mkdir devops → Creates a folder called devops.

pwd: Shows the current directory you’re in.

Example: pwd → Output: /home/user/devops

touch file.txt: Creates an empty file named file.txt.

Example: touch file.txt → Creates file.txt.

clear: Clears the terminal screen.

cd foldername: Changes to the folder named “foldername”.

Example: cd devops → Moves to the devops folder.

cd ..: Goes back one directory.

Example: If you’re in /home/user/devops, cd .. takes you to /home/user.

rmdir foldername: Deletes an empty folder.

Example: rmdir devops → Deletes the empty devops folder.

rm file.txt: Deletes a file.

Example: rm file.txt → Deletes file.txt.

rm -rf foldername: Deletes a folder and all its contents (files and subfolders).

Example: rm -rf devops → Deletes the devops folder and everything inside it.

cat file.txt: Displays the contents of a file.

Example: cat file.txt → Output: hello world (if file.txt contains “hello world”).

echo "hello world" > file.txt: Writes “hello world” to file.txt, overwriting it.

Example: echo "hello world" > file.txt → file.txt now contains “hello world”.

echo "how are u" >> file.txt: Appends “how are u” to file.txt without overwriting.

Example: echo "how are u" >> file.txt → Adds “how are u” to the end of file.txt.

zcat file.gz: Displays the contents of a compressed (zipped) file without unzipping it.

Example: zcat file.gz → Shows the contents of file.gz.

head file.txt: Shows the first 10 lines of a file.

Example: head file.txt → Displays the first 10 lines of file.txt.

tail file.txt: Shows the last 10 lines of a file.

Example: tail file.txt → Displays the last 10 lines of file.txt.

tail -n 5 file.txt: Shows the last 5 lines of a file.

Example: tail -n 5 file.txt → Displays the last 5 lines.

more file.txt: Shows a file page by page (press space to see the next page).

Example: more file.txt → Displays file.txt one screen at a time.

less file.txt: Similar to more, but you can scroll up and down.

Example: less file.txt → Opens file.txt for scrolling.

cp file1.txt file2.txt: Copies file1.txt to file2.txt.

Example: cp file1.txt file2.txt → Creates a copy of file1.txt named file2.txt.

cp -r folder1 folder2: Copies a folder and its contents.

Example: cp -r cloud devops → Copies the cloud folder to a new folder named devops.

mv file1.txt file2.txt: Moves or renames a file.

Example: mv file1.txt file2.txt → Renames file1.txt to file2.txt.

wc file.txt: Counts words, lines, and bytes (size) in a file.

Example: wc file.txt → Output: 10 50 300 file.txt (10 lines, 50 words, 300 characters).

ln file.txt hardlink-file: Creates a hard link to a file (a shortcut that stays even if the original file is deleted).

Example: ln file.txt hardlink-file → Creates a hard link to file.txt.

ln -s file.txt softlink-file: Creates a soft link (a shortcut that breaks if the original file is deleted).

Example: ln -s file.txt softlink-file → Creates a soft link to file.txt.

cut -b 1-4 file.txt: Extracts a portion of text (bytes 1 to 4) from each line of a file.

Example: cut -b 1-4 file.txt → Output: “this” (if file.txt contains “this is my file”).

tee file.txt: Writes output to both the screen and a file.

Example: echo "hello" | tee file.txt → Shows “hello” on the screen and writes it to file.txt.

sort file.txt: Sorts the lines of a file alphabetically.

Example: sort file.txt → Sorts the lines in file.txt.

diff file1.txt file2.txt: Shows differences between two files.

Example: diff file1.txt file2.txt → Lists differences between file1.txt and file2.txt.

vim file.txt: Opens a file in the Vim editor for editing.

Example: vim file.txt → Opens file.txt for editing.

ssh user@ipaddress: Connects to a remote server securely using SSH.

Example: ssh -i key.pem ubuntu@192.168.1.1 → Connects to a server with IP 192.168.1.1 as the user “ubuntu” using a key file.

df -h: Shows disk space usage in a human-readable format.

Example: df -h → Output: Shows used and available disk space.

du -h foldername: Shows the size of a folder or file.

Example: du -h devops → Output: Shows the size of the devops folder.

ps: Shows running processes for the current session.

Example: ps → Lists processes running in your terminal.

top: Shows live system resource usage (CPU, RAM, processes).

Example: top → Displays a live view of running processes and system usage.

fuser file.txt: Shows which process is using a file.

Example: fuser file.txt → Lists the process ID using file.txt.

kill 1234: Stops a process with the given process ID (e.g., 1234).

Example: kill 1234 → Terminates process ID 1234.

free -h: Shows memory (RAM) usage in a human-readable format.

Example: free -h → Output: Shows used and free RAM.

nohup command: Runs a command that keeps running even if you close the terminal.

Example: nohup free -h & → Runs free -h in the background and saves output to nohup.out.

vmstat: Shows virtual memory statistics.

Example: vmstat -a → Shows active and inactive memory.

12. System-Level Commands

These commands give information about the system:

uname: Shows the operating system name (e.g., Linux, Ubuntu).

Example: uname → Output: Linux

uptime: Shows how long the system has been running and the load average.

Example: uptime → Output: 03:00:00 up 9 min, 1 user, load average: 0.01

who: Lists all users currently logged into the system.

Example: who → Output: Lists usernames and login times.

whoami: Shows the current logged-in user.

Example: whoami → Output: ubuntu

which command: Shows the location of a command.

Example: which bash → Output: /bin/bash

id: Shows the user ID and group ID of the current user.

Example: id → Output: uid=1000(ubuntu) gid=1000(ubuntu)

sudo: Runs a command with superuser (admin) privileges.

Example: sudo apt install docker → Installs Docker with admin rights.

cat /etc/group: Shows the groups on the system.

Example: cat /etc/group → Lists all groups and their members.

shutdown: Shuts down the system.

Example: sudo shutdown now → Shuts down immediately.

reboot: Restarts the system.

Example: sudo reboot → Restarts the system.

13. Package Managers

Package managers install, update, or remove software on Linux.

apt: Used on Ubuntu/Debian to manage packages.

Example: sudo apt install docker → Installs Docker.

Example: sudo apt remove docker → Uninstalls Docker.

yum, dnf, pacman:

yum: Used on CentOS for package management.

dnf: Used on Fedora, an improved version of yum.

pacman: Used on Arch Linux.

Example: sudo yum install docker → Installs Docker on CentOS.

14. Users and Groups

Linux is a multi-user system, meaning many users can use it at once with different permissions.

useradd: Creates a new user.

Example: sudo useradd -m vaseem → Creates a user “vaseem” with a home directory.

passwd: Sets a password for a user.

Example: sudo passwd vaseem → Sets a password for user “vaseem”.

su: Switches to another user.

Example: su vaseem → Switches to the user “vaseem”.

userdel: Deletes a user.

Example: sudo userdel vaseem → Deletes the user “vaseem”.

groupadd: Creates a new group.

Example: sudo groupadd devops → Creates a group named “devops”.

gpasswd: Adds users to a group.

Example: sudo gpasswd -a ajay devops → Adds user “ajay” to the devops group.

groupdel: Deletes a group.

Example: sudo groupdel devops → Deletes the devops group.

15. File Permissions

Linux files and folders have permissions for the owner, group, and others.

Permissions:

r (read) = 4: Allows reading a file or listing a folder.

w (write) = 2: Allows modifying a file or folder.

x (execute) = 1: Allows running a file (like a script) or entering a folder.

chmod: Changes file or folder permissions.

Example: chmod 777 file.txt → Gives full permissions (read, write, execute) to everyone.

umask: Sets default permissions for new files.

Example: umask 0002 → New files will have permissions like 664 (rw-rw-r--).

chown: Changes the owner of a file or folder.

Example: sudo chown ajay file.txt → Makes “ajay” the owner of file.txt.

chgrp: Changes the group of a file or folder.

Example: sudo chgrp devops file.txt → Sets the group of file.txt to “devops”.

16. Compression Commands

These commands compress or decompress files and folders.

zip: Compresses files/folders into a .zip file.

Example: zip -r backup.zip folder → Compresses the folder into backup.zip.

unzip: Extracts a .zip file.

Example: unzip backup.zip → Extracts the contents of backup.zip.

tar: Creates or extracts .tar archives.

Example: tar -cvf file.tar folder → Creates a file.tar archive from the folder.

Example: tar -xvf file.tar → Extracts the contents of file.tar.

gzip/gunzip: Compresses or decompresses files.

Example: gzip file.txt → Compresses file.txt to file.txt.gz.

Example: gunzip file.txt.gz → Decompresses file.txt.gz.

17. File Transfer Commands

These commands transfer files between local and remote systems.

scp: Securely copies files between local and remote systems.

Example: scp -i key.pem file.txt ubuntu@192.168.1.1:/home/ubuntu → Copies file.txt to a remote server.

rsync: Synchronizes files between local and remote systems, only transferring changes.

Example: rsync -e "ssh -i key.pem" -avz file.txt ubuntu@192.168.1.1:/home/ubuntu → Syncs file.txt to a remote server.

18. Networking Commands

These commands help manage and troubleshoot networks.

ping: Checks if a server or website is reachable.

Example: ping youtube.com → Shows if youtube.com responds.

netstat: Shows network connections and statistics.

Example: netstat → Lists active connections and ports.

ifconfig: Shows network interface details (like Ethernet or Wi-Fi).

Example: ifconfig → Lists interfaces like eth0 (Ethernet) or lo (loopback).

traceroute: Shows the path data takes to reach a server.

Example: traceroute youtube.com → Lists all hops (routers) between you and youtube.com.

tracepath: Similar to traceroute but simpler.

Example: tracepath sigmasoftinfotech.com → Shows the path to sigmasoftinfotech.com.

mtr: Combines ping and traceroute.

Example: mtr sigmasoftinfotech.com → Shows live ping and path data.

nslookup: Finds the IP address of a domain.

Example: nslookup sigmasoftinfotech.com → Shows the IP address and server details.

telnet: Tests if a server is reachable on a specific port.

Example: telnet sigmasoftinfotech.com 80 → Checks if port 80 is open.

hostname: Shows or sets the system’s hostname.

Example: hostname → Output: mycomputer

Example: sudo hostname newname → Changes the hostname to “newname”.

ip: Shows IP address and network details.

Example: ip address show → Lists all network interfaces and their IP addresses.

iwconfig: Shows wireless network details.

Example: iwconfig → Lists Wi-Fi interfaces and details (if installed).

ss: Shows socket statistics (similar to netstat).

Example: ss → Lists active network connections.

dig: Gets detailed DNS information for a domain.

Example: dig sigmasoftinfotech.com → Shows DNS records.

whois: Shows who owns a domain and its registration details.

Example: whois sigmasoftinfotech.com → Shows domain owner and registrar.

arp: Shows the Address Resolution Protocol table (maps IP addresses to MAC addresses).

Example: arp → Lists devices on the network.

ifplugstatus: Checks if network interfaces are connected.

Example: ifplugstatus → Shows if eth0 is plugged in.

curl: Makes requests to APIs or downloads data from a URL.

Example: curl -X GET https://api.example.com → Fetches data from an API.

wget: Downloads files from the internet.

Example: wget https://example.com/file.txt → Downloads file.txt.

iptables: Manages firewall rules.

Example: sudo iptables --list-rules → Shows current firewall rules.

watch: Runs a command repeatedly and shows its output.

Example: watch -n 5 top → Runs top every 5 seconds.

nmap: Scans a network for open ports and devices.

Example: nmap sigmasoftinfotech.com → Shows open ports on sigmasoftinfotech.com.

route: Shows or modifies the routing table.

Example: route → Lists routing information.

19. Pro Linux Commands (awk, sed, grep)

These are advanced commands for processing text and logs.

awk:

A powerful tool to process and extract data from files.

Example: awk '{print $1,$2}' app.log → Prints the first two columns (e.g., date and time) from app.log.

Example: awk '/INFO/ {print}' app.log → Prints lines containing “INFO”.

Example: awk '/INFO/ {count++} END {print count}' app.log → Counts how many times “INFO” appears.

Example: awk '$2 >= "08:53:00" && $2 <= "08:53:59" {print}' app.log → Prints logs between 8:53:00 and 8:53:59.

sed:

A stream editor to edit text in real-time.

Example: sed -n '/INFO/p' app.log → Prints lines containing “INFO”.

Example: sed 's/INFO/LOG/g' app.log → Replaces “INFO” with “LOG” in app.log.

Example: sed -n -e '/INFO/=' app.log → Shows line numbers where “INFO” appears.

Example: sed '1,15 s/INFO/LOG/g;1,10p;11q' app.log → Replaces “INFO” with “LOG” in lines 1-15, prints lines 1-10, and stops at line 11.

grep:

Searches for text patterns in files.

Example: grep "INFO" app.log → Shows lines containing “INFO”.

20. Role of a DevOps Engineer

A DevOps engineer deploys and maintains applications on servers.

They ensure applications run smoothly, troubleshoot issues, and manage servers (often Linux).

Example: If an application like YouTube stops working, they:

Check if the server is reachable (ping).

Check logs for errors (cat, grep, awk).

Restart the server or service if needed (systemctl or kill).

Ensure proper permissions (chmod, chown) and network settings (iptables, netstat).

Devops-blog

Computer Networking for devops

How does the internet work ?

Data-link + physical layer combines = Network access layer

Hin-English Explanation

Full English Explanation

Kubernates Basic to Advance

Monolithic and microservice.

Kubernates Architecture:

master node mey kuch services rehthey hey

Setup On Local:

Steps to implement HPA:

Stress Testing

Cluster Administration :

RBAC( Role based Acess control):

Verify Roles and RoleBinding

Verify ServiceAccounts

Test Access Using Impersonation

4. Setting Up the Kubernetes Dashboard

This Helm chart deploys an Apache HTTP Server on a Kubernetes cluster.

Prerequisites

Chart Structure

Download Istio

Install the Kubernetes Gateway API CRDs

Deploy the sample application

Open the application to outside traffic

Access the application

View the dashboard

Uninstall

Projects:

Jenkins in One shot .

Introduction to Jenkins & CI CD

Jenkins setup on Vm :

Declarative pipelines:

Agents :

Declarative pipeline:

Shared Libraries:

User management in jenkins (Role based)

CI/CD project three tiwe with k8s , argocd , prometheus ,devsecops: sonarqube , trivy , owasp and email notification on pipeline pass/ fail.

Wanderlust Mega Project End to End Implementation

In this demo, we will see how to deploy an end to end three tier MERN stack application on EKS cluster.

Project Deployment Flow:

Tech stack used in this project:

How pipeline will look after deployment:

Pre-requisites to implement this project:

Setting up jenkins worker node

Steps to add email notification

Nodes Creation:

Steps to implement the project:

ab mujhey jenkins ko aur sonar ko integrate karana hey

argo cd :

How to monitor EKS cluster, kubernetes components and workloads using prometheus and grafana via HELM (On Master machine)

Clean Up

Docker in One Shot

Agenda:

Daigramatically :

Docker Architecture kya hey :

docker images:

Docker Networking :

Docker volume and storage:

Docker Compose:

Docker Registry :

mulsti-stage Docker builds

Monitoring and logging in docker

Orchestrating docker with kubernates (Introduction)

Projects:

project hey - web app hey (django mey likhey hey ) vo nginx ser server hoga - djnango ka backend hey - vo data base sey connect hoga mysql ka

Three tier application with docker compose java app

Docker scout

Docker init:

Git and Github For DevOps

Agenda:

mainey ek file banayi jiska name demo.py - jiskey andar ek mainey function banaliya

ae wrong code hey agar mai git status karun , git add and commit kardun , ideally

this file is now addes , thabki ae file galath file hey

mai isko pre commit hooks sey theek kar detha hun

pip install flake8

is tool sey code ka quality check karliya , ae code quality test kartha hey

kya mai as a devops engineer , ae tool ko apney git key process mey daldun

commit honesey pehley ae tool chal jaye