diff --git a/README.md b/README.md index 2eb0b66..5c76416 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,75 @@ -[TOC] +# Docker Begins -#### Disabled options -#### Run a container:busybox, start PID1 process + +### Installations +> docker + +> docker-compose + +> docker-machine + +1. find your daemon +2. docker info + +#### docker desktop for mac + +> Find symlinks `find $(which docker) -type l -ls` + +> Mac native application, that you install in /Applications `find /var/run/docker.sock -type l -ls` + +> hyperkit does not use docker-machine to create VM. docker-machine and docker desktop for mac can co-exist. + +> gives only one VM to run a docker daemon + +### docker toolbox on mac +``` +docker-machine create -d "virtualbox" project1 +docker-machine env project1 +docker-machine create -d "virtualbox" project2 +docker-machine env project2 +eval $(docker-machine env project1) +``` +> notice docker_host URL. IP address for each VM is different + +##### switch between docker desktop and docker toolbox + +``` +unset ${!DOCKER_*} #stop using docker-machine if any +eval $(docker-machine env project1) #set env variables to use a docker-machine +``` +[SKIP] +##### Run docker command on mac but connect to docker daemon running in docker-machine +`docker -H tcp://192.168.99.102:2376 --tlsverify --tlscert /Users/shadjachaudhari13/.docker/machine/machines/project1/cert.pem --tlscacert /Users/shadjachaudhari13/.docker/machine/machines/project1/ca.pem --tlskey /Users/shadjachaudhari13/.docker/machine/machines/project1/key.pem --tls ps` +[SKIP] + +##### To SSH into docker-machine +``` +eval $(docker-machine env project1) +docker run -d -p 8000:80 nginx # browse :8000 on mac's browser +docker-machine ssh project1 +ping # works +docker run -d nginx +ps aux | grep nginx +ping # works without exposing ports +``` + +#### Run a container:busybox and start first process - PID 1 syntax `docker run ` + ```bash docker run busybox echo hello docker run busybox cat /etc/hostname ``` -[container exits as PID 1 has done its job and terminated] + +> container exits as PID 1 has done its job and terminated + ``` docker run busybox sleep 300 containerid=$(docker ps | awk 'NR > 1 {print $1}') docker top $containerid ``` -[PID 1 would be 'sleep 300'] +> PID 1 would be 'sleep 300' + #### Exit foreground container in different ways exit ctrl+c @@ -22,8 +78,18 @@ Interactive containers are useful when you are putting together your own image. ``` docker run --interactive --tty ubuntu apt-get update && apt-get install telnet git -y +ctrl+c ``` -[stops container by exiting --tty] +> stops container by exiting --tty + +[TIP] +#### Daemonize foreground container +``` +docker run -it ubuntu +ctrl+p Ctrl+q +``` +> tty exits but container still runs in background + #### Run container in background - detached mode ``` docker run -d ubuntu @@ -35,21 +101,24 @@ docker exec --interactive --tty $containerid bash apt-get update ``` #### Exit background container in different ways -kill PID 1 and exit are not same -docker stop -#### Daemonize foreground container -``` -docker run -it ubuntu -ctrl+p Ctrl+q -``` -[tty exits but container still runs in background] +ctrl+c +type 'exit' + +#### stop container +1. docker stop +2. docker -it exec bash # stopping PID1 is stopping container + $ kill 1 + #### Write dockerfile and build image: nginx with telnet,git - upper case for DIRECTIVEs -- each Directive is considered as difference in file system, is run in an intermediate container. notice docker build log lines: - running in intermediate container - removing intermediate container - using cache -- Because of portability reasons, folders on host CAN'T be mounted in Dockerfile because the Dockerfile should run anywhere, it shouldnt be dependent on host. + +[SKIP] +- each Directive is considered as difference in file system(RUN ADD COPY contribute to layer creation), is run in an intermediate container. notice docker build log lines: + 1. running in intermediate container + 2. removing intermediate container + 3. using cache +NOTE: Because of portability reasons, folders on host CAN'T be mounted in Dockerfile because the Dockerfile should run anywhere, it shouldnt be dependent on host. +[SKIP] ``` docker build . -f-< -f-< access 192.168.99.100:9000 in REST client + +> note build context + +#### COPY +> COPY . . will always invalidate cache if file being copied are changed. +> COPY src dest +> COPY copies files along with their permissions +> src is relative to Dockerfile, dest is absolute path inside container. +> Can't COPY from parent directory COPY ../ . is invalid. use WORKDIR to set $(pwd) + +#### WORKDIR +> sets current working directory for instructions RUN, CMD, ENTRYPOINT, COPY and ADD while building Dockerfile +> WORKDIR(pwd for all subsequent instructions IN Dockerfile) and build context(is where Dockerfile is) are different from each other +> WORKDIR will be created if not existing already, could be a blank folder too +> useful if the executable/artifacts you want to run when container starts are located somewhere in subdirectories +> not recommended to use RUN cd .. like commands +> keep app files in separate dir than root, avoid clutter + +[build context continued..] +``` cd ~/Downloads docker build . -f-< note build context. Current directory and sub directories are set to build context. keep only required files along with Dockerfile + +> make changes to nodefun repository foo=bar1 + +##### cache: reuse existing layers +[run previous command twice to observe cached lines] ``` -cd .. -mkdir testDir && cd testDir docker build . -f-< sample output + +Sending build context to Docker daemon 125kB +Step 1/4 : FROM node:8 + ---> 1f6c34f7921c +Step 2/4 : RUN git clone https://github.com/shadjachaudhari13/nodefun + ---> *Using cache* + ---> 1eef8c4feaca +Step 3/4 : WORKDIR nodefun + ---> Using cache + ---> 28f94c66a848 +Step 4/4 : RUN npm install + ---> Using cache + ---> ff3438c4324d +Successfully built ff3438c4324d ``` -[note build context is couple of KBs. Current directory and sub directories are set to build context. keep only required files along with Dockerfile] -##### cache: reuse existing layers + +TIP: +1. share code from local +2. set remote context +3. keep instructions that are likely to bring bring change at the end of file +4. trick the Dockerfile!!! (add instruction that will invalidate cache at correct moment) + ``` docker build . -f-< hack -# not using cache, as cache got invalidated by previous instruction +FROM node:8 +RUN echo $(date)>trick.log RUN git clone https://github.com/shadjachaudhari13/nodefun +WORKDIR nodefun +RUN npm install EOF ``` + #### Push/Pull image:tag to repository ``` -docker tag shadjachaudhari/nginx:mydemo -docker push shadjachaudhari/nginx:mydemo -docker tag shadjachaudhari/nginx -docker push shadjachaudhari/nginx +docker tag shadjachaudhari/nodefun:mydemo +docker login +docker push shadjachaudhari/nodefun:mydemo +docker tag shadjachaudhari/nodefun +docker push shadjachaudhari/nodefun ``` -[when tag not mentioned latest tag is applied,replaced] +> docker pull fetches independent layers in parallel + +> when tag not mentioned latest tag is applied,replaced + ##### Use your own image as base image ``` docker build . -f-< share code repo on the host with docker container, dynamic changes to app and test quickly + +> store gems/requirements in volume and share such volume with app container. this way dependencies can be cached and reused while only new ones will be added/removed + +> VOLUME directive in dockerfile makes sure a mount point is initialized when container runs + +> containers can share volumes with each other + +> sharing data is not same as copying data into containers + + +# Docker Compose Rises + + +#### docker compose volumes: use case: dependencies in separate container from app ``` -docker run -d -v /Users/shadjachaudhari13/my-github/default.conf:/etc/nginx/conf.d/default.conf shadjachaudhari/nginx:mydemo -docker commit shadjachaudhari/nginx:mynewdemo -docker run -it shadjachaudhari/nginx:mynewdemo bash -cat /data/default.conf +git clone https://github.com/shadjachaudhari13/rubyfun.git +cd rubyfun +docker-compose -f docker-compose-v1.yaml up --build ``` -#### docker compose volumes: quick look +> source code shared at /rubyfun + +> dependency are getting installed everytime. can they be reused? ``` -git clone https://github.com/shadjachaudhari13/rubyfun.git -docker-compose build -docker-compose up +docker-compose -f docker-compose-v2.yaml up --build +docker-compose -f docker-compose-v2.yaml up --build +docker volume ls +docker images ``` -[volume "bundle" is shared by container "box" at path /box and container "app" at path /gems. Note that the mount paths are different] +> name of docker image built by docker-compose +Note: ruby image is 900mb. Lets save it offline. -#### COPY ``` -docker build . -f-< -docker commit shadjachaudhari/nginx:mynewdemo -docker run -it shadjachaudhari/nginx:mynewdemo cat /etc/nginx/conf.d/default.conf +docker save -o rubyfun.tar +docker rmi -f $(docker images -q) +docker-compose -f docker-compose-v2.yaml up --build # kill process when you see layers are being pulled freshly +docker load -i rubyfun.tar +docker-compose -f docker-compose-v2.yaml up # all layers exists ``` -- COPY src dest -- COPY copies files along with their permissions -- src is relative to Dockerfile, dest is absolute path inside container -#### WORKDIR -- sets current working directory for instructions RUN, CMD, ENTRYPOINT, COPY and ADD while building Dockerfile -- WORKDIR and build context are different from each other -- WORKDIR will be created if not existing already, could be a blank folder too -- useful if the executable/artifacts you want to run when container starts are located somewhere in subdirectories -- not recommended to use RUN cd .. like commands -- keep app files in separate dir than root, avoid clutter -[goblin demo] + #### ENV - variables used in Dockerfiles -[PORT node-fun demo] -#### Modify app dynamically using volumes: node-fun -[foo=bar foo=tar] +[PORT nodefun ARG vs ENV demo] (https://github.com/shadjachaudhari13/nodefun/blob/master/Dockerfile) + +#### Modify app dynamically using volumes: nodefun +[Dynamic update demo] (https://github.com/shadjachaudhari13/nodefun/blob/master/index.js) + +``` +https://github.com/shadjachaudhari13/nodefun.git +cd nodefun +docker-compose up --build --force-recreate +# change index.js +docker-compose up --force-recreate + ##### give up sudo privilleges -[demo uid] +[why not to run container as root](./uid-demo-Dockerfile) ``` sudo chmod 600 secrets.txt docker build -f uid-demo-Dockerfile --no-cache . docker run ``` -#### write docker compose for single container app -docker-compose ps -docker-compose down/up/build -docker-compose volumes -build dockerfile in some other directory -[write docker compose for node-fun] #### write docker compose with networks -[demo nginx with host network and keep changing PORT] -[demo docker-compose networks] -- multiple networks in one compose file -- one container can get multiple IPs as it can belong to multiple networks -[docker-compose network yaml] -#### How to build parent image? -Build from scratch: your first layer of fs -debootstrap to tar +[demo nginx with host network] (./docker-compose-nginx.yaml) +[keep changing PORT] (./default.conf) +[demo docker-compose networks] (./docker-compose-networks.yaml) +> multiple networks in one compose file +> one container can get multiple IPs as it can belong to multiple networks + +#### How to build parent image? Ways to create images? +> Build from scratch: your first layer of fs + +> debootstrap to tar example https://github.com/tianon/docker-brew-ubuntu-core/blob/185c5e23efaa8c7c857683e6dcf4d886acda3cba/trusty/Dockerfile -## Installations -docker -docker-compose -docker-machine (from docker toolbox) -### docker toolbox on mac -``` -docker-machine create -d "virtualbox" project1 -docker-machine env project1 -docker-machine create -d "virtualbox" project2 -docker-machine env project2 -eval $(docker-machine env project1) -``` -- notice docker_host URL. IP address for each VM is different -##### Run docker command on mac but connect to docker daemon running in docker-machine -`docker -H tcp://192.168.99.102:2376 --tlsverify --tlscert /Users/shadjachaudhari13/.docker/machine/machines/project1/cert.pem --tlscacert /Users/shadjachaudhari13/.docker/machine/machines/project1/ca.pem --tlskey /Users/shadjachaudhari13/.docker/machine/machines/project1/key.pem --tls ps` -##### To SSH into docker-machine -``` -docker-machine ssh project1 -ifconfig #Notice docker0 interface -docker run -d nginx -ping 192.168.99.102:80 # without publishing port on container -ping 172.17.0.2 -``` -#### docker desktop for mac -- Find symlinks -`find $(which docker) -type l -ls` -- Mac native application, that you install in /Applications -`find /var/run/docker.sock -type l -ls` -- hyperkit does not use docker-machine to create VM. docker-machine and docker desktop for mac can co-exist. -- gives only one VM to run a docker daemon -##### switch between docker desktop and docker toolbox -``` -unset ${!DOCKER_*} #stop using docker-machine if any -eval $(docker-machine env project1) #set env variables to use a docker-machine -``` +> docker commit shadjachaudhari/rubyfun:latest diff --git a/Technowise-Meetup-Notes.md b/Technowise-Meetup-Notes.md new file mode 100644 index 0000000..ce07788 --- /dev/null +++ b/Technowise-Meetup-Notes.md @@ -0,0 +1,108 @@ +# Technowise Docker Meetup Notes + +#### + +`$ git clone https://github.com/shadjachaudhari13/nodeshark.git` + +`$ cd nodeshark` + +`$ docker run node:10-alpine` + +`$ docker ps # see no running containers` + +`$ docker ps -a. # see one stopped container` + +`$ docker run node:10-alpine sh` # runs process 'sh' once container is running, exits once process is complete + +`$ docker ps` # see no running containers + +`$ docker ps -a` # see two stopped containers + +`$ docker run -it node:10-alpine sh` # this is your first running {container 1} + +#### + +`$ pwd` # nodeshark directory from Line 4 + +`$ docker ps # get ID of a running container + +`$ docker cp . :/.` # copy contents from current dir nodeshark to container root filesystem at / + +#### + +`$ ls` # contents copied can be seen here + +`$ pwd` # should show just /. as its a the root filesystem for container + +`$ npm install` # to fetch dependencies using package.json + +`$ export PORT=5050` + +`$ node app.js` # app is now running at port 5050 + +#### + +`$ docker exec -it sh` # opens door to already running container + +`$ wget http://localhost:5050` # index.html is downloaded + +`$ exit` # come out of container, this also stops the container + +`$ docker rm ` # removes the stopped container + +"CMD node app.js" add this as last line in crude.Dockerfile + +`$ docker build -f crude.Dockerfile .` + +`$ docker run -d -p 5050:5050 .` # this is {container 2} + +[access application in web browser localhost:5050] + +`$ docker exec -it sh` + +`$ whoami` # show "root" default user is root unless specified otherwise + +`$ ls` # contents of nodeshark dir + +`$ pwd` # shows just "/" + +`$ uname -a` # shows alpine linux distribution + +`$ printenv` # show PORT as environment variable set + +`$ ls node_modules` # node packages downloaded by "npm install" + +`$ exit` + +[Let's build same dockerfile again.] + +`$ docker build -f crude.Dockerfile .` + +[all build log lines say "using cache"] + +add "RUN ls -la node_modules" before "CMD node app.js" in crude.Dockerfile + +`$ docker build -f crude.Dockerfile .` + +`$ docker build . --no-cache` + +`$ docker build ` + +`$ docker tag :` + +`$ docker tag 12345678 your-name/nodeshark:mydemo` + +`$ docker push your-name/nodeshark:mydemo` + +`$ which docker` # see what you installed where you installed + +`$ git clone https://github.com/shadjachaudhari13/expense-tracker-nodejs.git` + +`$ cd expense-tracker-nodejs` + +`$ docker-compose up` + +`$ docker-compose -f docker-compose-with-persist-data.yaml up` + +`$ docker-compose -f docker-compose-node-dependencies.yaml up` + diff --git a/docker-build.log b/docker-build.log new file mode 100644 index 0000000..b34a15f --- /dev/null +++ b/docker-build.log @@ -0,0 +1,43 @@ +shadjas-MacBook-Pro:nodeshark shadjachaudhari13$ docker build . --no-cache +Sending build context to Docker daemon 97.28kB +Step 1/10 : FROM node:10-alpine + ---> fe6ff768f798 +Step 2/10 : RUN mkdir -p /home/node/app + ---> Running in d5808723721a +Removing intermediate container d5808723721a + ---> 1e35a5e2d923 +Step 3/10 : WORKDIR /home/node/app + ---> Running in 32647a1ae86a +Removing intermediate container 32647a1ae86a + ---> 02b037d0d169 +Step 4/10 : COPY package*.json ./ + ---> 69cf8591e44b +Step 5/10 : RUN npm install + ---> Running in 01b3ed40e6e5 +npm notice created a lockfile as package-lock.json. You should commit this file. +npm WARN nodejs-image-demo@1.0.0 No repository field. + +added 48 packages from 36 contributors and audited 121 packages in 2.275s +found 0 vulnerabilities + +Removing intermediate container 01b3ed40e6e5 + ---> c8991ca2dd09 +Step 6/10 : COPY . . + ---> 960384a51d38 +Step 7/10 : ENV PORT 5000 + ---> Running in 96daae9603fc +Removing intermediate container 96daae9603fc + ---> 640e7640f1f9 +Step 8/10 : ARG PORT=5000 + ---> Running in 1b3f5ed38e8e +Removing intermediate container 1b3f5ed38e8e + ---> 11c9c6a95c1a +Step 9/10 : EXPOSE $PORT + ---> Running in 68ad5f69661f +Removing intermediate container 68ad5f69661f + ---> 32c1f39e8276 +Step 10/10 : CMD [ "node", "app.js" ] + ---> Running in 3d20f60058e6 +Removing intermediate container 3d20f60058e6 + ---> 473b4d6cd2f5 +Successfully built 473b4d6cd2f5