-
-
Notifications
You must be signed in to change notification settings - Fork 251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New Shipping Apps section #377
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
# Shipping | ||
|
||
_Ahoy, matey!_ | ||
Oh! you've built the greatest app ever and, of course, using Crystal!! | ||
And now you want to share it with the whole world?! | ||
|
||
Well, it’s your lucky ~~pirate~~ day because we are going to ship our first Crystal application!! | ||
|
||
So, _weigh anchor and hoist the mizzen! Yarr!_ | ||
|
||
## The App | ||
|
||
The application we are shipping is an example of a static file sharing server. Here’s the source code: | ||
|
||
```crystal | ||
# staticserver.cr | ||
require "http" | ||
require "option_parser" | ||
|
||
# Handle Ctrl+C and kill signal. | ||
# Needed for hosting this process in a docker | ||
# as the entry point command | ||
Signal::INT.trap { puts "Caught Ctrl+C..."; exit } | ||
Signal::TERM.trap { puts "Caught kill..."; exit } | ||
|
||
path = "/www" | ||
port = 80 | ||
|
||
option_parser = OptionParser.parse do |parser| | ||
parser.on "-f PATH", "--files=PATH", "Files path (default: /www)" do |files_path| | ||
path = files_path | ||
end | ||
parser.on "-p PORT", "--port=PORT", "Port to listen (default: 80)" do |server_port| | ||
port = server_port.to_i | ||
end | ||
end | ||
|
||
server = HTTP::Server.new([ | ||
HTTP::LogHandler.new, | ||
HTTP::ErrorHandler.new, | ||
HTTP::StaticFileHandler.new(path), | ||
]) | ||
|
||
address = server.bind_tcp "0.0.0.0", port | ||
puts "Listening on http://#{address} and serving files in path #{path}" | ||
server.listen | ||
``` | ||
|
||
So, starting the server (listening on port 8080 and serving files under the current directory) is as easy as running: | ||
|
||
```shell-session | ||
$ crystal ./src/staticserver.cr -- -p 8080 -f . | ||
Listening on http://0.0.0.0:8080 and serving files in path . | ||
``` | ||
|
||
**Note:** the default behavior is to listen on `port 80` and serve the folder `/www`. | ||
|
||
## Compiling our application | ||
|
||
Let’s go over Crystal’s [introduction](https://crystal-lang.org/reference/). One of the main goals of the language is to _Compile to efficient native code_. That means that each time that we compile our code then an executable is built, but with an important property: it has a target platform (architecture and operating system), which is where the application will run. Crystal knows the target platform because is the same as the one being used to compile. | ||
For example, if we use a Linux OS based computer for compiling, then the executable will be meant to run on a Linux OS (and in some cases we will need to use the same Linux distribution). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that without further setting up the environment the binary can't be dropped on any linux distro. We played safe maybe. But on different distros paths of libraries might change and would require the user to set There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure, but that's not restricted by the distro, but the specific environment setup. You should be able to run a Linux executable on any Linux system (i.e. not needing to use the same distro), assuming the environment is configured appropriately. |
||
|
||
Can we set the target when calling the compiler? Oh, that’s a great idea, but for now it’s not an option (there are a lot of great buccanears [working on a solution](https://forum.crystal-lang.org/t/cross-compiling-automatically-to-osx/1330/12) and remember that Crystal is open source: so you are welcome aboard!) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This section suggest you can't cross compile Crystal. That is very much possible, though. Only linking the final executable is not covered for a cross-compile target. I'd either rephrase this paragraph or maybe even remove it entirely? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree to rephrase it. Cross compiling can be mention as something possible as long as you are aware of the difference between compiling, linking. But I would not aim for an explanation. At most that currently you would need a target environment to perform the linking. |
||
|
||
Let’s compile our application: | ||
|
||
```shell-session | ||
$ shards build --production | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not about There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like that shards create a nicer directory structure. But either we show the shards.yml and the structure of the whole folder. But since we are simulating a whole app, using shards makes sense to me. Otherwise, it feels we are not using our own tools. |
||
Dependencies are satisfied | ||
Building: staticserver | ||
``` | ||
|
||
and now, if we want to know the file type: | ||
|
||
If we are using Mac OS, we will see something like this: | ||
|
||
```shell-session | ||
$ file bin/staticserver | ||
bin/staticserver: Mach-O 64-bit executable x86_64 | ||
``` | ||
|
||
And if we are using a Linux distribution, for example Ubuntu: | ||
|
||
```shell-session | ||
$ file bin/staticserver | ||
bin/staticserver: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=a78ffe59325c3f9668d551852e7717e3996edb3b, not stripped | ||
``` | ||
|
||
|
||
Furthermore, our application may use some libraries (our application’s dependencies!) and so the target platform should have this libraries installed. To list the libraries used by our application: | ||
|
||
On Mac OS: | ||
|
||
```shell-session | ||
$ otool -L ./bin/staticserver | ||
./bin/staticserver: | ||
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11) | ||
/usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0) | ||
/usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0) | ||
/usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0) | ||
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1) | ||
/usr/local/opt/libevent/lib/libevent-2.1.7.dylib (compatibility version 8.0.0, current version 8.0.0) | ||
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) | ||
``` | ||
|
||
On a Linux OS we may use `ldd bin/staticserver` with a similar result. | ||
|
||
Up to this point, we know that for shipping our application, we need to compile for each target platform where we want our application to run; and also, we need to provide the dependencies used by our application. | ||
|
||
Here we will see two ways for shipping our application: | ||
|
||
* Using a [Docker](https://www.docker.com/get-started) image. | ||
* Using a [Snapcraft](https://snapcraft.io/build) package. | ||
|
||
|
||
## Shipping with Docker | ||
|
||
The idea behind using Docker is to create a Docker container, with a target platform, and use it for building our application and then create a really small image for shipping and running our application! | ||
|
||
Wow! I want to [embark on this adventure](./shipping/docker.html)! | ||
|
||
## Shipping with Snapcraft | ||
|
||
The idea behind using Snapcraft is to use this tool for building an executable targeting the Linux OS and then publishing it! | ||
|
||
Oh great! Let’s follow [this sea lane](./shipping/snapcraft.html)! |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
# Shipping with Docker | ||
|
||
Already at the docks?! Not at all, matey, we’ve just started the journey! Docker will let us ship our application in a more homogeneous way. | ||
|
||
We will create a `Dockerfile` that will let us: | ||
* create a base image | ||
* build the application | ||
* list the dependencies | ||
* build a custom docker image (only with our application and the dependencies) based on a small docker image. | ||
|
||
And then we will publish the final image (with our application) | ||
|
||
## Creating the Dockerfile | ||
|
||
Let’s create a `Dockerfile`. This file will create a docker image from other image: `crystallang/crystal:latest`, which is based on Ubuntu and it ships with the Crystal compiler. On the other hand, we will use multistage-build, so that our new image will be as small as possible (brave buccaneers have [already sailed](https://manas.tech/blog/2017/04/03/shipping-crystal-apps-in-a-small-docker-image/) these [wild seas](https://gist.github.com/bcardiff/85ae47e66ff0df35a78697508fcb49af)). | ||
|
||
Our `Dockerfile` will look like this: | ||
|
||
```dockerfile | ||
FROM crystallang/crystal:latest | ||
|
||
ADD . /src | ||
WORKDIR /src | ||
RUN shards build --production | ||
|
||
RUN ldd bin/staticserver | tr -s '[:blank:]' '\n' | grep '^/' | \ | ||
xargs -I % sh -c 'mkdir -p $(dirname deps%); cp % deps%;' | ||
|
||
FROM scratch | ||
COPY --from=0 /src/deps / | ||
COPY --from=0 /src/bin/staticserver /staticserver | ||
|
||
EXPOSE 80 | ||
|
||
ENTRYPOINT ["/staticserver"] | ||
``` | ||
|
||
**Note:** if you are building an application that needs static files (for example: you are building a Web Application and you need `favicon.ico`), in that case we would need to copy those files to the final image using `COPY`. | ||
|
||
Let’s build it with: | ||
|
||
```shell-session | ||
$ docker build -t "staticserver:0.1.0" . | ||
``` | ||
|
||
Was our image created? Well, I hope so (or someone will be _walking the plank_) | ||
To be certain, let’s run: | ||
|
||
```shell-session | ||
$ docker images | ||
REPOSITORY TAG IMAGE ID CREATED SIZE | ||
staticserver 0.1.0 0b57eeef751c 9 seconds ago 10.4MB | ||
``` | ||
|
||
Sink Me!! Only 10.4MB! This is great! | ||
|
||
**Why are we listing the dependencies inside the container?** | ||
|
||
We are doing this because the dependencies would be different depending on the Operating System the application is running (or will run). In this case we need the dependencies for an Ubuntu distribution. | ||
|
||
**And why are we using multistage-build?** | ||
|
||
Well, first we need the Crystal compiler for building our application, so we base our image on the `crystallang/crystal:latest`image. | ||
Then we won’t need the compiler anymore, so we base the final image on Docker Official Image [scratch](https://hub.docker.com/_/scratch/). In case the `scratch` image is not enough then we may use another image based on Ubuntu since the Crystal image is based on this Linux distribution. | ||
|
||
**Wait! and if I’m using an external library like `sqlite`?** | ||
|
||
Oh well, in that case we are going to need it for compiling and then the script should make the external library available in the final image. | ||
|
||
Continuing our example, before publishing our image let’s see if it’s working: | ||
|
||
```shell-session | ||
$ docker run --rm -it -v ${PWD}:/www -p 8080:80 staticserver:0.1.0 | ||
Listening on http://0.0.0.0:80 and serving files in path /www | ||
``` | ||
|
||
If we go to our browser and navigate to `http://localhost:8080` then we will see the files list. Yarr! | ||
|
||
### Publishing | ||
|
||
Finally, we only have to publish our new image with our application! | ||
To do so, we may use [`docker push`](https://docs.docker.com/engine/reference/commandline/push/) to push the image to a registry (for example [Docker Hub](https://hub.docker.com/)) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# Shipping with snapcraft.io | ||
|
||
Snapcraft will allow us to package and publish our application for users using Linux. Also, it will allow the users to discover our application more easily through the [Snapcraft Store](https://snapcraft.io/store) | ||
We only need to create a configuration file where we set the language, the application and dependencies. So let’s start! | ||
|
||
## Creating the snap | ||
|
||
As the [snapcraft documentation](https://snapcraft.io/docs) says _Snaps are app packages for desktop, cloud and IoT that are easy to install, secure, cross-platform and dependency-free_ And also important: they are **containerised** software. | ||
|
||
To describe our application, Snapcraft uses a [snapcraft.yaml](https://snapcraft.io/docs/snapcraft-yaml-reference) file. In our example, the file `snapcraft.yaml` will be located in a folder called `snap` and it will look like this: | ||
|
||
```yaml | ||
name: crystal-staticserver | ||
version: "0.1.0" | ||
summary: Create the static file server snap | ||
description: Create the static file server snap | ||
|
||
base: core | ||
grade: devel | ||
confinement: classic | ||
build-packages: | ||
- libz-dev | ||
- libssl-dev | ||
|
||
apps: | ||
crystal-staticserver: | ||
command: bin/staticserver | ||
|
||
parts: | ||
crystal-staticserver: | ||
plugin: crystal | ||
source: ./ | ||
``` | ||
|
||
Let’s see some of the fields: | ||
|
||
`name`, `version`, `summary` and `description` define and describe our application, allowing users to easily find software in the store. | ||
|
||
The `base` field will let us specify a [base snap](https://snapcraft.io/docs/base-snaps) which will _provide a run-time environment with a minimal set of libraries_. | ||
In our example `base: core` is based on `Ubuntu 16.04 LTS` | ||
|
||
The `grade` field could be `stable` or `devel` (for development). This will have an impact on the [channels](https://snapcraft.io/docs/channels) where our application could be published. | ||
|
||
The `confinement`field will set the [degree of isolation](https://snapcraft.io/docs/snap-confinement) | ||
|
||
In the `build-packages` field we may list needed libraries for building our application. | ||
|
||
In the `apps` field we will set `app-name` and the `command` to run it. | ||
|
||
The `parts` field will let us define the differents [building blocks](https://snapcraft.io/docs/snapcraft-parts-metadata) that form our application. | ||
Here, in the `plugin` field we may set the tool that will drive the building process. In our example we will use the [crystal plugin](https://snapcraft.io/docs/the-crystal-plugin) | ||
|
||
Great! Now, we only need to build the package, using: | ||
|
||
```shell-session | ||
$ snapcraft | ||
Launching a VM. | ||
Starting snapcraft-crystal-staticserver - | ||
... | ||
Snapping 'crystal-staticserver' \ | ||
Snapped crystal-staticserver_0.1.0_amd64.snap | ||
``` | ||
|
||
## Publishing | ||
|
||
Finally, to share our application to the world, we need to publish it in the Snapcraft Store. Follow the steps described in the [official documentation](https://snapcraft.io/docs/releasing-your-app) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.