Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Shipping Apps section #377

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,3 +132,6 @@
* [Hosting on GitLab](guides/hosting/gitlab.md)
* [Continuous Integration](guides/continuous_integration.md)
* [Using TravisCI](guides/ci/travis.md)
* [Shipping Apps](guides/shipping.md)
* [Using Docker](guides/shipping/docker.md)
* [Using Snapcraft](guides/shipping/snapcraft.md)
126 changes: 126 additions & 0 deletions guides/shipping.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Shipping

_Ahoy, matey!_
Oh! you've built the greatest app ever and, of course, using Crystal!!
And now you want to share it with the whole world?!

Well, it’s your lucky ~~pirate~~ day because we are going to ship our first Crystal application!!

So, _weigh anchor and hoist the mizzen! Yarr!_

## The App

The application we are shipping is an example of a static file sharing server. Here’s the source code:

```crystal
# staticserver.cr
require "http"
require "option_parser"

# Handle Ctrl+C and kill signal.
# Needed for hosting this process in a docker
# as the entry point command
Signal::INT.trap { puts "Caught Ctrl+C..."; exit }
Signal::TERM.trap { puts "Caught kill..."; exit }

path = "/www"
port = 80

option_parser = OptionParser.parse do |parser|
parser.on "-f PATH", "--files=PATH", "Files path (default: /www)" do |files_path|
path = files_path
end
parser.on "-p PORT", "--port=PORT", "Port to listen (default: 80)" do |server_port|
port = server_port.to_i
end
end

server = HTTP::Server.new([
HTTP::LogHandler.new,
HTTP::ErrorHandler.new,
HTTP::StaticFileHandler.new(path),
])

address = server.bind_tcp "0.0.0.0", port
puts "Listening on http://#{address} and serving files in path #{path}"
server.listen
```

So, starting the server (listening on port 8080 and serving files under the current directory) is as easy as running:

```shell-session
$ crystal ./src/staticserver.cr -- -p 8080 -f .
Listening on http://0.0.0.0:8080 and serving files in path .
```

**Note:** the default behavior is to listen on `port 80` and serve the folder `/www`.

## Compiling our application

Let’s go over Crystal’s [introduction](https://crystal-lang.org/reference/). One of the main goals of the language is to _Compile to efficient native code_. That means that each time that we compile our code then an executable is built, but with an important property: it has a target platform (architecture and operating system), which is where the application will run. Crystal knows the target platform because is the same as the one being used to compile.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Let’s go over Crystal’s [introduction](https://crystal-lang.org/reference/). One of the main goals of the language is to _Compile to efficient native code_. That means that each time that we compile our code then an executable is built, but with an important property: it has a target platform (architecture and operating system), which is where the application will run. Crystal knows the target platform because is the same as the one being used to compile.
Let’s go over Crystal’s [introduction](https://crystal-lang.org/reference/). One of the main goals of the language is to _Compile to efficient native code_. That means that each time that we compile our code then an executable is built, but with an important property: it has a target platform (architecture and operating system), which is where the application will run. Crystal chooses the target platform by default as the same as the one being used to compile.

For example, if we use a Linux OS based computer for compiling, then the executable will be meant to run on a Linux OS (and in some cases we will need to use the same Linux distribution).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and in some cases we will need to use the same Linux distribution) - I'm not aware of any distribution-specific limitations. Do you have an example for that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that without further setting up the environment the binary can't be dropped on any linux distro. We played safe maybe. But on different distros paths of libraries might change and would require the user to set LD_LIBRARY_PATH before running the executable. I am not sure if on a plain Alpine any binary linked with glibc can run (Ref).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but that's not restricted by the distro, but the specific environment setup. You should be able to run a Linux executable on any Linux system (i.e. not needing to use the same distro), assuming the environment is configured appropriately.


Can we set the target when calling the compiler? Oh, that’s a great idea, but for now it’s not an option (there are a lot of great buccanears [working on a solution](https://forum.crystal-lang.org/t/cross-compiling-automatically-to-osx/1330/12) and remember that Crystal is open source: so you are welcome aboard!)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section suggest you can't cross compile Crystal. That is very much possible, though.

Only linking the final executable is not covered for a cross-compile target.

I'd either rephrase this paragraph or maybe even remove it entirely?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree to rephrase it. Cross compiling can be mention as something possible as long as you are aware of the difference between compiling, linking. But I would not aim for an explanation. At most that currently you would need a target environment to perform the linking.


Let’s compile our application:

```shell-session
$ shards build --production
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not about shards, so it rather only use crystal build.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that shards create a nicer directory structure.

But either we show the shards.yml and the structure of the whole folder.
Or we use crystal build (and update the samples below removing the bin).

But since we are simulating a whole app, using shards makes sense to me. Otherwise, it feels we are not using our own tools.

Dependencies are satisfied
Building: staticserver
```

and now, if we want to know the file type:

If we are using Mac OS, we will see something like this:

```shell-session
$ file bin/staticserver
bin/staticserver: Mach-O 64-bit executable x86_64
```

And if we are using a Linux distribution, for example Ubuntu:

```shell-session
$ file bin/staticserver
bin/staticserver: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=a78ffe59325c3f9668d551852e7717e3996edb3b, not stripped
```


Furthermore, our application may use some libraries (our application’s dependencies!) and so the target platform should have this libraries installed. To list the libraries used by our application:

On Mac OS:

```shell-session
$ otool -L ./bin/staticserver
./bin/staticserver:
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/local/opt/libevent/lib/libevent-2.1.7.dylib (compatibility version 8.0.0, current version 8.0.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
```

On a Linux OS we may use `ldd bin/staticserver` with a similar result.

Up to this point, we know that for shipping our application, we need to compile for each target platform where we want our application to run; and also, we need to provide the dependencies used by our application.

Here we will see two ways for shipping our application:

* Using a [Docker](https://www.docker.com/get-started) image.
* Using a [Snapcraft](https://snapcraft.io/build) package.


## Shipping with Docker

The idea behind using Docker is to create a Docker container, with a target platform, and use it for building our application and then create a really small image for shipping and running our application!

Wow! I want to [embark on this adventure](./shipping/docker.html)!

## Shipping with Snapcraft

The idea behind using Snapcraft is to use this tool for building an executable targeting the Linux OS and then publishing it!

Oh great! Let’s follow [this sea lane](./shipping/snapcraft.html)!
82 changes: 82 additions & 0 deletions guides/shipping/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Shipping with Docker

Already at the docks?! Not at all, matey, we’ve just started the journey! Docker will let us ship our application in a more homogeneous way.

We will create a `Dockerfile` that will let us:
* create a base image
* build the application
* list the dependencies
* build a custom docker image (only with our application and the dependencies) based on a small docker image.

And then we will publish the final image (with our application)

## Creating the Dockerfile

Let’s create a `Dockerfile`. This file will create a docker image from other image: `crystallang/crystal:latest`, which is based on Ubuntu and it ships with the Crystal compiler. On the other hand, we will use multistage-build, so that our new image will be as small as possible (brave buccaneers have [already sailed](https://manas.tech/blog/2017/04/03/shipping-crystal-apps-in-a-small-docker-image/) these [wild seas](https://gist.github.com/bcardiff/85ae47e66ff0df35a78697508fcb49af)).

Our `Dockerfile` will look like this:

```dockerfile
FROM crystallang/crystal:latest

ADD . /src
WORKDIR /src
RUN shards build --production

RUN ldd bin/staticserver | tr -s '[:blank:]' '\n' | grep '^/' | \
xargs -I % sh -c 'mkdir -p $(dirname deps%); cp % deps%;'

FROM scratch
COPY --from=0 /src/deps /
COPY --from=0 /src/bin/staticserver /staticserver

EXPOSE 80

ENTRYPOINT ["/staticserver"]
```

**Note:** if you are building an application that needs static files (for example: you are building a Web Application and you need `favicon.ico`), in that case we would need to copy those files to the final image using `COPY`.

Let’s build it with:

```shell-session
$ docker build -t "staticserver:0.1.0" .
```

Was our image created? Well, I hope so (or someone will be _walking the plank_)
To be certain, let’s run:

```shell-session
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
staticserver 0.1.0 0b57eeef751c 9 seconds ago 10.4MB
```

Sink Me!! Only 10.4MB! This is great!

**Why are we listing the dependencies inside the container?**

We are doing this because the dependencies would be different depending on the Operating System the application is running (or will run). In this case we need the dependencies for an Ubuntu distribution.

**And why are we using multistage-build?**

Well, first we need the Crystal compiler for building our application, so we base our image on the `crystallang/crystal:latest`image.
Then we won’t need the compiler anymore, so we base the final image on Docker Official Image [scratch](https://hub.docker.com/_/scratch/). In case the `scratch` image is not enough then we may use another image based on Ubuntu since the Crystal image is based on this Linux distribution.

**Wait! and if I’m using an external library like `sqlite`?**

Oh well, in that case we are going to need it for compiling and then the script should make the external library available in the final image.

Continuing our example, before publishing our image let’s see if it’s working:

```shell-session
$ docker run --rm -it -v ${PWD}:/www -p 8080:80 staticserver:0.1.0
Listening on http://0.0.0.0:80 and serving files in path /www
```

If we go to our browser and navigate to `http://localhost:8080` then we will see the files list. Yarr!

### Publishing

Finally, we only have to publish our new image with our application!
To do so, we may use [`docker push`](https://docs.docker.com/engine/reference/commandline/push/) to push the image to a registry (for example [Docker Hub](https://hub.docker.com/))
66 changes: 66 additions & 0 deletions guides/shipping/snapcraft.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Shipping with snapcraft.io

Snapcraft will allow us to package and publish our application for users using Linux. Also, it will allow the users to discover our application more easily through the [Snapcraft Store](https://snapcraft.io/store)
We only need to create a configuration file where we set the language, the application and dependencies. So let’s start!

## Creating the snap

As the [snapcraft documentation](https://snapcraft.io/docs) says _Snaps are app packages for desktop, cloud and IoT that are easy to install, secure, cross-platform and dependency-free_ And also important: they are **containerised** software.

To describe our application, Snapcraft uses a [snapcraft.yaml](https://snapcraft.io/docs/snapcraft-yaml-reference) file. In our example, the file `snapcraft.yaml` will be located in a folder called `snap` and it will look like this:

```yaml
name: crystal-staticserver
version: "0.1.0"
summary: Create the static file server snap
description: Create the static file server snap

base: core
grade: devel
confinement: classic
build-packages:
- libz-dev
- libssl-dev

apps:
crystal-staticserver:
command: bin/staticserver

parts:
crystal-staticserver:
plugin: crystal
source: ./
```

Let’s see some of the fields:

`name`, `version`, `summary` and `description` define and describe our application, allowing users to easily find software in the store.

The `base` field will let us specify a [base snap](https://snapcraft.io/docs/base-snaps) which will _provide a run-time environment with a minimal set of libraries_.
In our example `base: core` is based on `Ubuntu 16.04 LTS`

The `grade` field could be `stable` or `devel` (for development). This will have an impact on the [channels](https://snapcraft.io/docs/channels) where our application could be published.

The `confinement`field will set the [degree of isolation](https://snapcraft.io/docs/snap-confinement)

In the `build-packages` field we may list needed libraries for building our application.

In the `apps` field we will set `app-name` and the `command` to run it.

The `parts` field will let us define the differents [building blocks](https://snapcraft.io/docs/snapcraft-parts-metadata) that form our application.
Here, in the `plugin` field we may set the tool that will drive the building process. In our example we will use the [crystal plugin](https://snapcraft.io/docs/the-crystal-plugin)

Great! Now, we only need to build the package, using:

```shell-session
$ snapcraft
Launching a VM.
Starting snapcraft-crystal-staticserver -
...
Snapping 'crystal-staticserver' \
Snapped crystal-staticserver_0.1.0_amd64.snap
```

## Publishing

Finally, to share our application to the world, we need to publish it in the Snapcraft Store. Follow the steps described in the [official documentation](https://snapcraft.io/docs/releasing-your-app)