You can find the repository for this project on GitHub

If you're anything like me, you've probably wondered before: "What kind of magic makes Composer go?" - You look at the source code of vendor/autoload.php and see it links to yet another autoloader that's called autoload_real.php Then there's a bunch of hashes and it's confusing and eventually you just accept that Composer is magic and you aren't worthy of its secrets.

Today, let's look at how you can build your own Composer. At least the autoloading part. We won't talk about the seven million other features that Composer provides to us aside from that.

Setting up the project

This time, all you need to bring is any kind of PHP installation. Technically it needs to be at least PHP 5.3, but I very much hope that you aren't running anything that old.

As always, we'll start with what we want our project structure to look like

psr-4/
├─ autoloader.php
├─ app.php
├─ src/
│  ├─ Pretzel/
│  │  ├─ RandomClass.php

autoloader.php is going to be what it says. This file will be roughly equivalent to vendor/autoload.php when using Composer.

app.php is going to be our test file where we include the autoloader and try to use our classes.

src/ will contain our application-specific code. The folder doesn't have to be Pretzel. You pick whatever makes you happy.

Our stated goal is to load every class that is located in src/Pretzel. Each subdirectory constitutes a new namespace. So any file in src/Pretzel/Factory has a namespace of Pretzel\Factory.

The magic spice: spl_autoload_register

All we really need to create an autoloader is that function: spl_autoload_register. You can read more details about it in the PHP documentation.

You call it like this

<?php

spl_autoload_register(function(string $class) {
    // Perform unknowable magic here
});

All you need to pass in is a callable of some kind that receives one argument: a string with a class name. You can register multiple of these callbacks too. This is useful if you want to have a few logically different ways to load classes.

After your function is registered, PHP will call it every time a class is used for the first time. This is usually either when you call new on it, or when you use a static function on it. Let's see that in action.

Interacting with spl_autoload_register

I'll set up my autoloader.php like this:

// autoloader.php

<?php

spl_autoloader_register(function(string $class) {
    die($class)
}

For now we'll just print the class name that was desired and exit. We also need a class to load, so let's build a quick PretzelFactory. It returns a pretzel emoji from a static method. It's very fancy.

// src/Pretzel/Factory/PretzelFactory

<?php

namespace Pretzel\Factory;

class PretzelFactory {
    public static function getPretzel(): string
    {
        return '🥨';
    }
}

We need to use that class somewhere. This is what our app.php file is for. We'll have to include our autoloader, pull the class in and then call the static method. Something like this.

// app.php

<?php

require __DIR__ . '/autoloader.php';

use Pretzel\Factory\PretzelFactory;

echo PretzelFactory::getPretzel();

When we run our app.php file, this is the result.

$ php app.php
Pretzel\Factory\PretzelFactory

So we get the fully-qualified class name (FQCN) passed to us. Now PSR-4 states that a FQCN name consists of:

• A namespace prefix (e.g. Pretzel) that corresponds to a base directory (e.g. src/Pretzel)
• Zero or more sub-namespaces (e.g. Factory) that correspond to a directory (e.g. src/Pretzel/Factory)
• One class name corresponding to a PHP file with that same name (e.g. PretzelFactory.php)

Some of you might have realized by now, that our FQCN is essentially a direct path to the file we need to load. How convenient!

Loading the appropriate files

To load our file, first we need to turn our FQCN into an actual path. This can be done by replacing every backslash with a forward slash and appending .php at the end. With that, we can already autoload files from a hard-coded base path!

Let's try it out!

// autoloader.php

<?php

function fqcnToPath(string $fqcn) {
    return str_replace('\\', '/', $fqcn) . '.php';
}

spl_autoload_register(function (string $class) {
    $path = fqcnToPath($class);

    require __DIR__ . '/src/' . $path;
});

After adding this, we can just run our app.php script again

$ php app.php
🥨

Great success! This shows you what autoloading boils down to: A fancy way to require a PHP file just in time when it's needed. But if we want to be (reasonably) spec-compliant, we still have work to do.

Right now the path we load from is hard-coded to be src/ but according to PSR-4, every package is free to define its own namespace prefix that maps to a path. This is essentially what you do every time you set up autoloading in Composer

{
    "autoload": {
        "psr-4": {
            "Pretzel\\": "src/Pretzel"
        }
    }
}

So let's try to parse this JSON snippet and use it to load our classes. I'll call it conductor.json, because that's what the logo of Composer depicts, and we're basically a cheap knock-off. 🤪

Autoloading according to configuration

The first thing we have to do is to grab our JSON file and turn it into something we can actually work with (i.e. an associative array). Luckily, PHP has native functions for both of those things.

// autoloader.php

<?php

$configuration = json_decode(
    file_get_contents('conductor.json'),
    true
);

$namespaces = $configuration['autoload']['psr-4'];

// .. SNIP ..

Now, the user is free to specify as many namespaces and base directories as she wants, so we need to check if the prefixed/first namespace is available in our psr-4 associative array. We could use a foreach loop, but we can also make it a bit more efficient using a function called strtok().

// autoloader.php

<?php
// .. SNIP ..

spl_autoload_register(function (string $class) use ($namespaces) {
    $prefix = strtok($class, '\\') . '\\';

    // We don't handle that namespace.
    // Return and hope some other autoloader handles it.
    if (!array_key_exists($prefix, $namespaces)) return;

    $baseDirectory = $namespaces[$prefix];
});

What strtok does is tokenize the string according to a separator. If we pass in the backslash, it will return us everything up to the first backslash, excluding that. To correctly match our configuration, we just add it again ourselves.

For the performance nerds among you this makes sure that we find the proper base directory in O(1) time.

Then we check if this namespace prefix maps to a path in our configuration. If it does, great! If it doesn't, we just return from the function and hope that someone else handles this case. We can't do anything else with it.

The next step is to remove the prefixed namespace from our class name. This is because we know exactly what directory the prefix maps to, and we want to start building our final path from that. We can do this in our fqcnToPath function.

// autoloader.php

<?php
// .. SNIP ..

function fqcnToPath(string $fqcn, string $prefix) {
    $relativeClass = ltrim($fqcn, $prefix);

    return str_replace('\\', '/', $relativeClass) . '.php';
}

spl_autoload_register(function (string $class) use ($namespaces) {
    $prefix = strtok($class, '\\') . '\\';

    // We don't handle that namespace.
    // Return and hope some other autoloader handles it.
    if (!array_key_exists($prefix, $namespaces)) return;

    $baseDirectory = $namespaces[$prefix];
    $path = fqcnToPath($class, $prefix);
});

Once again PHP has our back! We use ltrim to remove the main namespace from the beginning of our FQCN. This gives us the "relative" class, in the same sense as having a relative path. The rest of the function stays the same.

At this point we have a base directory and a relative path going out from that base directory. So the final piece that's missing is trying to require the appropriate file!

// autoloader.php

<?php
// .. SNIP ..

spl_autoload_register(function (string $class) use ($namespaces) {
    $prefix = strtok($class, '\\') . '\\';

    // We don't handle that namespace.
    // Return and hope some other autoloader handles it.
    if (!array_key_exists($prefix, $namespaces)) return;

    $baseDirectory = $namespaces[$prefix];
    $path = fqcnToPath($class, $prefix);

    require $baseDirectory . '/' . $path;
});

At this point we can run our app.php again and see if it works as advertised.

$ php app.php
🥨

Great! So new can add PSR-4 autoloading configuration to our JSON file just as if it were Composer. Of course, one should probably add some more input validation to this code. Users could forget to specify a \ in their namespace. Or they could accidentally add a / in the base directory. But I'll leave that part to you, dear reader.

Closing thoughts

While this is a simplified example of what Composer does internally, it still shows you the general principles of PSR-4 autoloading and what goes into it. Once you break it down, it's really not as complex as it seems!

The main reason I even got the idea was due to a side project I'm silently hacking away on. I needed a simple way to load in PHP classes that define plugins, and well, here we are.

As always, the entire code for this blog post is available on GitHub.

TL;DR: This post describes the structure behind my recently published pretzelhands/base repository.

For any seasoned PHP developer, looking at the repository should be enough. If you're interested in the rationale, feel free to read along.

This base project is intended as the common ground for any future projects built as part of this blog. For small projects it can feel over the top to bring along a whole Laravel install when a much smaller framework would do.

It also ensures that no one needs to be deep into any one ecosystem, unless it is explicitly mentioned.

Anybody who has recently read my Telegram bot post will already recognize the project structure.

Requirements

• PHP 7.4, ideally PHP 8.0
• Composer

We explicitly do not need a local web server such as Apache or nginx. We'll use a special command to get a shareable link for our project.

Creating the project structure

As any modern PHP project should, the app uses Composer to define and install dependencies. So let's set that up.

$ composer init

And for the basic setup we only require one other dependency. That's vlucas/phpdotenv. We use it to load sensitive data from a .env file. Let's install it!

$ composer require vlucas/phpdotenv

Then I want to create a folder structure that looks something like this

base-project/
├─ vendor/
├─ composer.json
├─ composer.lock
├─ .env
├─ bootstrap.php
├─ src/
│  ├─ Pretzel/
│  ├─ helpers.php
├─ public/
│  ├─ index.php

In this structure we first have the files that Composer generates. The vendor/ folder for our dependencies and autoloader, the composer.json defining our project meta data and dependencies and the composer.lock file pinning our dependencies to specific versions.

.env holds our sensitive data that can change between your local computer and your actual server. An example would be an API key to access GitHub or a key that the application uses for encryption.

bootstrap.php pulls in the Composer autoloader and sets up the loading of the .env files. It's used for any setup tasks that always need to run. We will usually include it in the PHP files located in public/

src/Pretzel contains our application specific code. Anything that makes your idea come to live! The folder name is based on the namespace. This will defined later and you can change it to anything you like.

src/helpers.php contains small helper functions that we want to use every now and again. For example, we'll define a convenience function for getting values from our .env file.

public/ contains all the things we want to expose to the scary internet. In MVC-speak these are our controllers, but in a very low-tech way. Here we deal with user input and maybe return some JSON. Or render a template file.

Setting up autoloading

Composer automatically knows how to pull in external dependencies and make them available to our application. It does however not know what classes it should load from our own code. We need to configure that. To change that, we insert the following into our composer.json

"autoload": {
    "files": [ "src/helpers.php" ],
    "psr-4": {
        "Pretzel\\": "src/Pretzel"
    }
}

The first key, files, tells Composer specific files to load. This is useful if you have files that don't contain classes. And our helpers.php is just a collection of plain old functions.

The psr-4 key tells Composer which namespace to look for and where to look for it. Because I am always on-brand, I called my namespace Pretzel. Feel free to change this. We also tell Composer to look in the src/Pretzel directory for any classes it can find.

And since I always have trouble remembering this: It's namespace first, then the directory. The namespace has to end in an escaped backslash.

After you've made the changes, don't forget to run composer dump-autoload so the autoloader is aware of the new configuration.

Setting up our application with bootstrap.php

Whenever our application runs, there are certain things we want to do. We always want to load our .env file. We always want to load the Composer dependencies. If we have one, we always want to set up our library for database connections.

This all happens in bootstrap.php. For our basic project, all we need is 3 lines.

// bootstrap.php

<?php

require __DIR__ . '/vendor/autoload.php';

$dotenv = Dotenv\Dotenv::createImmutable([__DIR__]);
$dotenv->load();

The first require is a common sight for anyone who has dealt with Composer before. It loads all dependencies into our application.

Line 2 and 3 tell the vlucas/phpdotenv package to look in the current directory for a .env file and load it. If we don't have an .env file our application will yell at us about that and stop execution.

Adding the first helper

Now that we have set up the loading of our .env file, we have all the values contained in that file loaded into the $_ENV superglobal.

That's neat, but interacting with superglobals directly is generally considered a Bad Idea™. So instead, let's wrap our $_ENV variable into a nice env() function.

// src/helpers.php

<?php

if (!function_exists('env')) {
    function env($value, $default = null) {
        return $_ENV[$value] ?? $default;
    }
}

We have added the function_exists check, because some packages (e.g. illuminate/support) might bring along their own version of an env() function. If that's the case we don't want to overwrite it and just use the one they give us instead.

Inside the function we check if $_ENV holds the value we want. If it does not we use the null-coalescing operator ?? to get a default value. If there's no default value, we just return an empty value.

Making our application shareable

Often times when building projects, it can be useful to have a public endpoint that points to your local development environment. For example, when you're working with a third-party service that uses webhooks to notify you of events.

Thus it makes sense to make the application easily shareable.

For this, we need one more package. It's called expose and it's made by the cool people at BeyondCode. You can learn more about it on their website. Let's install it!

$ composer global require beyondcode/expose

But to share an application we need to have it be served by some kind of HTTP server. Many tutorials would go on a long tangent about setting up nginx or Apache here. For our little framework here, however, the built-in PHP server is plenty.

You call it like this:

$ php -S localhost:8080 -t public

Here the port can be anything you want. The -t option allows us to specify which directory is the public root of our application. In our case that's public/.

After running this you can open up your browser and look at localhost:8080 to see the result. Since we only have an empty directory at this point, you'll get a 404 error.

Now let's combine expose and the PHP server into one thing:

php -S localhost:8080 -t public &>/dev/null & expose share localhost:8080

This command first starts up the server and redirects all output to /dev/null. That means we're essentially throwing it away. The PHP server just logs whatever requests are coming in anyways. Expose does the same thing, so we don't need that output.

The & after the first command tells the PHP server to continue running in the background. This allows us to immediately run another command, which is expose!

We point it to serve whatever is happening at localhost:8080, which is where our application currently lives!

And since all that is a bit of a hassle to type, we can wrap it in a so-called Composer script. To do so, add the following to your composer.json

"scripts": {
    "dev": "php -S localhost:8080 -t public &>/dev/null & expose share localhost:8080"
}

You can rename dev to whatever makes the most sense to you. And now that we've added this, we can run

$ composer run dev

and immediately have a publically shareable link to our app. Neat!

Using the framework

Now that everything is set up, we can use this for a very contrived example of building an application. It'll just take a value from our .env file and output it with some extra text we generate in a class.

To get started, let's add a value to our .env

MY_BIGGEST_SECRET="I love puppies and kittens"

Next, let's create a PretzelFactory.php file in our src/ directory and add the following.

// src/PretzelFactory.php

class PretzelFactory
{
    public static function getPretzel()
    {
        return '🥨';
    }
}

This is not a factory in the true sense of the word, but it gives us an infinite supply of pretzels and that counts for something!

To wrap it up we create an index.php file in our public directory.

// public/index.php

// Our setup file
require __DIR__ . '/../bootstrap.php'; 

use Pretzel\PretzelFactory;

echo sprintf(
    '%s %s'
    PretzelFactory::getPretzel(),
    env('MY_BIGGEST_SECRET')
);

Here we read load in our bootstrap.php to get the app going. Then we load in the PretzelFactory class. Finally we output a formatted string containing a pretzel and our big secret.

And now we can share it like this

$ composer run dev

The result should look a little bit like this

Closing thoughts

This concludes the basic project I will be using for my future blog posts. Anyone who wants to have a pre-made copy of it can clone it from GitHub.

In the future when starting from this point I will be linking this blog post and the repository, instead of explaining the entire process every time.

Unless of course we diverge from it. Then there will be a proper explanation.

Personally I've been using it since 2015. Since 2018, I've used it for 95% of my daily communication. During that time I also found out that Telegram has a really cool Chat Bot API! So I've built a few.

• One keeps track of my appointments and sends reminders
• One tracks shipping goals for a chat group of my friends
• One that checked who in the group got made fun of the most
• One that tracks some metrics of my company (audience growth, ...)

Today, I'd like to teach you how to do the same. Because a future where we all have a personal army of automated chat robots is definitely one I want to live in!

Prerequisites

To follow along with this article you'll need only two things.

• A working installation of PHP 7.2+ and Composer
• The Telegram app and an account

To create our bot, we need a special token from Telegram itself. We can get this from an automated bot, too! It's called @BotFather and it will guide you through the setup process. So let's do that now!

Congratulations! You have set up your first Telegram bot. Be sure to note down that token from the last message. We'll need it in a few minutes. You can also further customize your bot with a profile image and such. I'll leave that up to you.

Setting up your PHP project

Now it's time to get down to the actual core of this project: Implementing the chat bot. We'll start as with any good PHP project in $CURRENT_YEAR: By setting up a Composer project.

$ composer init

While we could directly deal with the HTTP API of Telegram, there is a very nice library already built for us by Irfaq Sayed. So let's grab that too!

$ composer require irazasyed/telegram-bot-sdk

Then it's time to define our directory structure. When I build Telegram bots, I generally follow a structure that looks like this:

chat-bot/
├─ composer.json
├─ composer.lock
├─ vendor/
├─ src/
├─ public/
├─ .env
├─ bootstrap.php
├─ helpers.php
├─ setup.php

First we have our regular files as generated by Composer.

src/ is the folder where we place all of our commands and other application-specific code.

public/ will contain the code we expose to the scary, scary reality of the internet.

bootstrap.php will set up some things such as including our autoloader and loading variables from our .env file. We'll include that one a lot.

helpers.php usually contains small utility functions that don't fit anywhere else.

Last but not least, setup.php is a small file that helps us set up our connection with Telegram.

Now would be a good time to copy that token we got from BotFather earlier and put it in our .env file.

# .env
TELEGRAM_BOT_TOKEN="1234567890:YOUR_TOKEN"

To wrap up this setup sequence all that's left to is to set up our autoloader. To do this we modify our composer.json and add this:

"autoload": {
    "psr-4": {
        "Pretzel\\": "src/"
    },
    "files": [ "helpers.php" ]
}

The change essentially summarizes to this: Load everything from src/ into the Pretzel namespace, and please also load the helpers.php file specifically.

Now we finally have our PHP project ready to go!

Setting up the connection with Telegram

Telegram works with an event-based system to let you know when new updates arrive. For this purpose we need to set up a webhook handler. In real world terms this just a PHP file that Telegram sends a request to every time somebody messages our bot. Fancy!

First, we'll need our bot's token. That is how we identify to Telegram who we are. As you may remember we put that into a .env file. So let's set up loading variables from that.

$ composer require vlucas/phpdotenv

Then we add the following to our bootstrap.php

// bootstrap.php

<?php

require __DIR__ . "/vendor/autoload.php";

$dotenv = Dotenv\Dotenv::createImmutable(__DIR__);
$dotenv->load();

Thanks to that short and sweet snippet we can now easily grab values from our .env file by using the env() function. This is not a native PHP function, but the Telegram Bot SDK brings it along as a dependency.

Next, I want to set up a small helper function in our helpers.php

// helpers.php

<?php

function telegram(): \Telegram\Bot\Api
{
    return new \Telegram\Bot\Api(
        env('TELEGRAM_BOT_TOKEN')
    );
}

This is purely for convenience. It allows us to grab an instance of the Telegram API wrapper without constantly having to query the .env file again. Now we can combine those two things in our setup.php file.

// setup.php

<?php

require 'bootstrap.php';

telegram()->setWebhook(['url' => env('TELEGRAM_WEBHOOK')]);

echo "Setup the Telegram webhook!";

Hold your horses. Readers with sharp eyes will have noticed, that I'm getting a TELEGRAM_WEBHOOK from the .env file. We don't have that yet, right? Right.

We need to pass Telegram some kind of URL it can send its updates to. How do we do this on localhost? I'm glad you asked. Being a PHP developer in 2021 is a wonderful experience, so we have the perfect tool for this. It's called expose and it's made by BeyondCode.

Note: There are other options out there too. Something like ngrok would work just as well. I like expose because it's easy to set up and provides custom subdomains for free.

Setting up your local webhook

To get started with getting an URL for our local environment, first we install expose.

$ composer global require beyondcode/expose

This installs the expose utility globally and gives us access to it. If you've been using Composer for a while you've probably already done that, but make sure that composer global config bin-dir --absolute is in your $PATH.

Next we need to run our PHP somehow. For a project such as this the integrated PHP web server is fine. So we can use the following in the root directory of your project to get it running:

$ php -S localhost:8080 -t public

[Mon Jan 18 11:28:18 2021] PHP 8.0.0 Development Server (http://localhost:8080) started

Then, open another terminal tab and run the following:

$ expose share localhost:8080 --subdomain=pbbot

In the resulting output towards the end you should find a URL ending in sharedwithexpose.com. In my case it is pbbot.sharedwithexpose.com and that is the URL we need. Remember it!

Making it more ergonomic (optional)

Running two different terminal tabs at once is cumbersome. So let's make use of ✨magical Composer scripts✨ so we only have to run one command to get our bot up to speed. We can make use of the & functionality of the shell to run both things at the same time. So you can add this to your composer.json

"scripts": {
    "dev": "php -S localhost:8080 -t public &>/dev/null & expose share localhost:8080 --subdomain=pbbot"
}

The &>/dev/null stuff just ensures that the output of the original PHP web server does not mess with expose. Since expose gives us a nice request overview, we don't need it anyways. Now you can run a single command, and your bot comes alive!

$ composer run dev

Now we can add our webhook URL to our .env file

# .env

TELEGRAM_BOT_TOKEN="1234567890:YOUR_TOKEN"
TELEGRAM_WEBHOOK="https://pbbot.sharedwithexpose.com/webhook.php"

webhook.php will be the file we use to handle any incoming requests from Telegram. We'll create it in the next step. First, we want to run our little setup.php script.

$ php setup.php
Setup the Telegram webhook!

Setting up your webhook handler

So we just told Telegram to please send us any information to webhook.php, but we don't have that file yet. We'll just create it in our public/ folder. The webhook handler is essentially responsible for grabbing any unprocessed updates, forwarding them to our bot, and returning a result to Telegram. It looks like this:

// public/webhook.php

<?php

require __DIR__ . '/../bootstrap.php';

telegram()->commandsHandler(true);

I think by now you can see why the Telegram Bot SDK is useful. The fact that we have to call only one function to set up an entire command system is super useful and keeps productivity high!

Writing your first command

Now that we have all the annoying setup stuff out of the way, we can write our first command. Let's start with the most bare-bones command imaginable: When we say /hello to our bot, we want it to say Hello World back to us.

First, we have to create a new class for our command. I'll put it in src/Commands/HelloCommand.php, so we can access it through the class Pretzel\Commands\HelloCommand. Every command we create needs to inherit from the Bot SDKs base class. It also needs to have a name. So the basic structure looks like this.

// src/Commands/HelloCommand.php

<?php

namespace Pretzel\Commands;

use Telegram\Bot\Commands\Command;

class HelloCommand extends Command
{
    protected $name = 'hello';

    public function handle()
    {
        // TODO: Implement command
    }
}

Now inside the handle() function, we have access to a whole bunch of replyWith* type functions that we can use to respond to the message we just received. The easiest way is to use replyWithMessage.

These functions all take a bunch of arguments in the form of an array. I can't even tell you all of them by heart. Thankfully, there's good documentation! To find out what you need to send to the function, you can just look it up in the API Methods documentation. I also highly recommend the official Telegram documentation for more detailed information.

Each send* function corresponds to one of our replyWith* functions.

The important thing that these replyWith* functions do is adding the chat_id parameter to our arguments. So we can safely ignore that one. The other parameter we need is text. It should contain, as the name implies, the text of our message.

So now our class should look something like this

// src/Commands/HelloCommand.php

<?php

namespace Pretzel\Commands;

use Telegram\Bot\Commands\Command;

class HelloCommand extends Command
{
    protected $name = 'hello';

    public function handle()
    {
        $this->replyWithMessage([
            'text' => 'Hello World!'
        ]);
    }
}

Registering it with the bot

Now we need to register this command with the bot, so it actually knows what commands are available. To do this, we go back to our bootstrap.php file and add the following

// bootstrap.php

telegram()->addCommands([
    Pretzel\Commands\HelloCommand::class,
]);

And now that we have done this, we can interact with our bot! Open up the chat with your bot in Telegram and try sending it a /hello command. It should respond to you right away!

Working with arguments

Of course, commands are much more interesting when you're able to send arguments to them. Say we want to greet a friend! How do we pass in their name?

For this, the Command class we inherit from offers a $pattern property. Here we can either use named arguments in braces like {this}, or we can pass in a custom regex.

// src/Commands/HelloCommand.php

class HelloCommand extends Command
{
    protected $pattern = '{name}'; // Our argument is called 'name'
    protected $pattern = '{name}?'; // We have an optional argument called 'name'
    protected $pattern = '.+' // We'll take in any argument with 1 or more characters. It'll be called 'custom'

    // .. SNIP ..
}

So let's say we want our bot to greet a person by their name if it receives one. Else we just greet the world, because everyone should be greeted once in a while. For this we set our $pattern to {name}?.

Then, in our handle() function we can just grab the pre-supplied $arguments property:

// src/Commands/HelloCommand.php

<?php

class HelloCommand extends Command
{
    protected $name = 'hello';
    protected $pattern = '{name}?';

    public function handle()
    {
        $arguments = collect($this->arguments);

        // .. SNIP ..
    }
}

As you can see, I used a the collect function when getting the arguments. This is another beautiful thing from the Laravel world and brought along by the Bot SDK. Collections are basically super-powered PHP arrays with a much cleaner way of working with them.

Of course you can also just handle the $arguments array yourself, but I vastly prefer this approach.

To learn more about Laravel collections, you can read the excellent documentation. They're available as a standalone package and are a massive quality-of-life improvement.

Now we can grab the name key from our arguments, or return a default value if it's not there. Then we insert this argument into a string and send it back to the user. That looks like this

// src/Commands/HelloCommand.php

<?php

class HelloCommand extends Command
{
    protected $name = 'hello';
    protected $pattern = '{name}?';

    public function handle()
    {
        $arguments = collect($this->arguments);

        $this->replyWithMessage([
            'text' => sprintf(
                'Hello %s!',
                $arguments->get('name', 'World')
            )
        ]);
    }
}

After saving our command we can try it immediately! Go over to your bot and send it /hello <YOUR_NAME> and watch as it greets you with it's mechanically engineered happiness!

Sending unprompted messages

Of course bots only have limited use if they only reply to you when you ask for them. Luckily, we can also send messages without any prior command. Just because we feel like it. For this, we need the chat_id of the person we want to send the messages to, or the @username of the group we want to send messages to.

To find out your personal chat_id you can just contact @RawDataBot, who will tell you in the first message.

If you have a private group without a @username, you can also just add @RawDataBot to the group to find the chat_id of the group. It will be a negative number.

And then you can use the sendMessage function of the Telegram API object. As demonstration, let's add an unprompted message to our setup.php that we successfully set up our webhook.

// setup.php

<?php

require 'bootstrap.php';

$telegram = telegram();
$telegram->setWebhook(['url' => env('TELEGRAM_WEBHOOK')]);
$telegram->sendMessage([
    'chat_id' => env('PRETZELHANDS_CHAT_ID'),
    'text' => 'Successfully set up webhook'
]);

That was pretty easy, wasn't it?

Processing raw messages without commands

For some bots, we may want to process messages that are sent without command, just using the raw information that Telegram provides us. This is once again, relatively easy. After passing through the commands system, the SDK gives us an update object in the webhook handler. We can then use that to do whatever!

// public/webhook.php

<?php

<?php

require __DIR__ . '/../bootstrap.php';

$updates = telegram()->commandsHandler(true);

// Telegram\Bot\Objects\Update Object(
//     [items:protected] => Array(
//         [update_id] => 290647531
//         [message] => Array(
//             [message_id] => 81
//                 [from] => Array(
//                 [id] => 0000000000
//                 [is_bot] =>
//                 [first_name] => Richard
//                 [last_name] => Blechinger
//                 [username] => pretzelhands
//                 [language_code] => en
//             )
// 
//             [chat] => Array(
//                 [id] => 00000000000
//                 [first_name] => Richard
//                 [last_name] => Blechinger
//                 [username] => pretzelhands
//                 [type] => private
//             )
// 
//             [date] => 1610977759
//             [text] => abc
//         )
//     )
// )

All properties on this update are directly accessible. So you can use $update->message->text to grab the relevant messages text etc.

This is the process I used in another bot to parse date and event information directly from a users message. Aside from the fact that parsing natural language dates is painful, it works great!

Deployment

Deployment for this setup should be straightforward for anyone who has ever dealt with other PHP deployments before. Doing so is left as an exercise for the reader, but it boils down to three points

• Push your files onto your server
• Point your webserver to serve files from public/
• Update webhook URL to yourdomain.com/webhook.php using setup.php

After that you're good to go!

Closing thoughts

This article should have given you enough of an overview to get started with writing your own Telegram bots for whatever purposes. From here on out, the possibilities are mostly limited to what you can imagine.

My main bot currently tracks newsletter growth, Twitter growth, and during Notebag's lifecycle it tracked the daily amount of money earned. It's like a little daily dashboard informing me how my side-project life is going. The second bot makes sure I show up to any appointments I have on time.

The code for this project is also available on GitHub

Test, test.. 1, 2. Is this thing on? Wonderful! Welcome back, everybody. Somewhere back in 2019 I had a month where I blogged a lot. I wrote most of the articles you see on here and many of them are actually quite popular! There's more than a thousand people stopping by here every month to learn about shell scripting from me.

I find that very silly, but hey. Glad that I can help some of you with your questions. And I'd like to do that again.

In the past two years I've learned a lot about programming. There's so many things I still haven't shared about payments and e-commerce that I learned while working for Wirecard.

In 2021 I have taken a conscious decision to step away from taking on more freelance work than I currently have. Instead I want to focus more on my own projects again. To build things that are interesting, helpful, or just funny to me. And part of that is engaging with people through this blog.

What's new?

Since its humble beginnings, this blog has gone through three major revisions. The third one being the current one. Here's a short overview:

• Two-file PHP setup that rendered Markdown on demand
• Custom PHP setup that rendered Markdown and Mustache
• Custom PHP static site generator from Markdown and Mustache

I didn't intend to build a static site generator, but the pieces were all just lying there and so I rewired them. Along with making this site super fast, I also added these things:

• Dedicated post series for multi-part things
• A weekly newsletter you can subscribe to
• After many requests: an RSS feed for you to consume
• Easier sharing to various places
• An actual navigation

And then I reworked how posts look and feel a little bit, which gives me the ability to write more structured and informative content.

For example I can now use this box to give more detailed context-specific information about various topics. Or snarky side comments.

Neat.

What's next?

With this announcement, the blog is officially considered "re-launched" by me and while I have a few topics in mind, I'd love to hear what you want to see me write about! Because it's more fun to write about stuff that is actually interesting to people.

If you want to help decide, please vote in this Twitter poll and I'll get on it for next week's post! Until then, stay safe, stay healthy and have a good time!

So. Here we are. The irony of my last two blog posts being about the start and end of something is not lost on me. I know that a few of you are probably confused, angry and perhaps even a bit sad right now, and I apologize in advance. And some critical voices that have been haunting me around the internet have finally been proven right.

Now let me tell you why exactly I am stopping work on Notebag, what will happen to it and where I will be going from here.

The Why

One or two of you reading this might perhaps know what it's like when you start a new project. You come up with a concept! Ideas burst through your head! You are itching to tear the world apart and make things come alive exactly the way you imagined them. This is an absolute high of productivity and you hack away until late at night, only to get up early next morning and get right back into it.

That was me.

That's how Notebag started, doubly fueled by the fact that the world in 2020 is .. something else and I was shit-out-of-luck in terms of freelancing contracts. So I clung to the one thing I had and put all of my energy into it and I worked to make it into something I thought was nice.

I pushed and pushed each day. Bigger! Better! More features!

And this worked well for a while, until the inevitable tiredness came creeping in. So I put Notebag on the backburner. New freelancing contracts finally started trickling in and I found myself busy with two kinds of work: The one paying my bills and the one I did on Notebag.

It was a struggle. Every day after work I tried to pour some of my remaining energy into Notebag, but nothing happened. I failed to implement the most basic things and bit by bit, this intense feeling of guilt set in.

In time, I started avoiding Twitter. Because everytime I tweeted, I felt like someone out there would be judging me. Why wasn't I reporting progress on Notebag? What was I even up to? And so I withdrew to the confines of Telegram and ignored Twitter. DMs came in asking and I was too scared to reply.

Worst of all, I was and am building this subconscious pressure on myself that I am not really allowed to work on anything else. Why would I work on Project X when Notebag fans are clamoring for updates? That makes me a bad person, no? This has led me to a state where I was afraid to program outside of my contracts. Not a great spot to be in.

When you are not happy with things, you should change them. I am not happy with how Notebag is currently holding a soft stranglehold on my life and so I am changing it.

The What

What does all of this mean for Notebag's future? First of all, I have published the source code. It's MIT-licensed and on GitHub, free for everyone to download, build and play with. In my wildest dreams of course there would be a shining community hero that rises up and picks up where I left off. But that is unlikely to happen.

So at least as a consolation, Notebag in its entirety is available for everyone, for all time, for free. If someone is seriously willing to make pull requests and improve the code, I am happy to release signed builds with those changes included. If anyone has specific questions about the architecture and why my code is so bad, I will gladly answer them.

This is, I think, the most elegant solution. It at least guarantees that the code doesn't just gather dust on my machine, forever unreachable for any interested person.

And since there is still the occasional sale of a license, I offer anybody who bought their license in the past 30 days (after September 27) a full refund, no questions asked. To avail this, just send me an email or send me a Twitter DM.

The Future

Notebags future probably does not hold many further improvements. I still use it as my personal preferred note-taking app, because that was its purpose. It works for me and my workflow and I hope that other people will keep on using it or maybe learn a thing or two from the source code.

Personally I only have one or two side projects, but I'll probably put them on indefinite hold. Clean slate. There's a few other hobbies I've been meaning to get around to and I think right now is a good time to do that. And perhaps, somewhere down the line an idea will strike me and I'll give in to the itch to build again. But the time for that is not right now.

Final words

I would again like to apologize to anyone who put high hopes into this app. Turns out that the detractors who pointed out my legacy of abandoned apps were onto something! It sucks to be known for this, but maybe one day that will change. Or maybe it won't.

As it is, Notebag will keep working in its usual state for the foreseeable future. But maybe Au revoir, doesn't have to mean goodbye forever. Not that I have any hopes of picking it back up, but if 2020 has taught the world anything, it's that life is unpredictable.

For my final point I'd like to give a special shout-out to @Clo__S, who aside from being an early adopter and wonderful source of feedback was probably Notebag's most vocal fan. Thanks for all the support along the way.

I was never any good at coming up with ideas. I'm not sure whether you could attribute it to a lack of creativity, my life being "boring" by many peoples standards or whether I've just been satisfied with existing solutions to all of my discomforts and problems.

Well, except for that one thing where I never found a satisfying solution: Note taking. I tried a few different approaches. For a while I would put my notes into my IDEs scratch file feature. That wasn't exactly a very scalable or searchable solution, but it was good enough for storing code snippets in.

For another while I had my writings in Apple Notes. I liked the simplicity of it, but it didn't have support for typing out notes in Markdown, which made formatting cumbersome.

Then I tried note taking wunderkind Bear, which is beloved by many. And for a while I was very happy with it. It has a wonderful set of features and great user experience. Seriously, hats off to the people at Bear. But I still had to reach for my mouse to do things every now and then. And there was still the thing where I had to either open the app on demand or switch around all my open windows with Cmd-Tab. It still felt off.

This entire journey went on for a good year or so. I had fought off the idea of writing a note taking app because I felt that:

• A) It would be super complicated
• B) The market for note taking apps is incredibly over saturated

I still believe that the second point holds true to a certain extent. There is certainly a lot of competition out there. But the first point was shattered when I found out about a lovely little thing called tiptap. It's based on ProseMirror which is a wonderful library for building rich text editors.

And on March 7, 2020 I finally got fed up enough to start writing code. (Although there were earlier indicators of what was about to happen).

Development

Notebag is an Electron-based app since I am primarily a web developer and native code seems scary. I would have loved to give Swift a try, but in the end stuck to what I know best. For the frontend I landed on Vue since that is what Tiptap uses and I don't really hold much of a strong opinion in the great war of the frontend frameworks.

One great thing that allowed development to move quickly is Prosemirrors concept of input rules. These are essentially little functions that take a regex you want to search for and then spit out appropriate HTML for the thing you just entered. This is what powers all of the Markdown formatting as well as a few custom functionalities such as the nestable categories and Zettelkasten-type links.

For most of March I was still working on this very sporadically as I was pre-occupied with contracting work for most of my week. Sadly due to the world going a bit haywire I ended up having not so much contract work anymore by April at which point I started focusing on the development in earnest.

Within a week or so I had implemented most of the barebones features of a note taking app (funnily enough, switching between notes was one of the last ones. Oops!) And I sent out my first beta builds. At this point the app looked a bit like a boring Apple Notes clone.

This is what I sent out to a few trusted beta testers

Differentiating the app

This was also the time at which I started drilling down on what I consider the unique selling points of the app. At first there is the omnibar or "Go To Anything". This is essentially the same thing as in any modern IDE or code editor.

You have a full fuzzy search for your notes. And this is also where the keyboard focus really took overhand. I spent a good chunk of the next two weeks assigning tabindexes to every important element, writing countless :hover, :active, :focus styles and adding numerous keyboard bindings that you can use to get around the app.

There was lots of talking with beta testers and refining and implementing and polishing. You never expect how much there is to do until you start writing it down and ending up with a todo list in the mid double digits.

How do I even marketing

Once the app reached a reasonably stable level of maturity, I had to get around to the point I am the worst at. The one I had pushed away again and again for as long as I could: I had to build a landing page to market this thing. Oh dear

The struggles started with naming. In total I went through 27 names before the right one appeared. Some of them were: Keynote, Typemark, Markflow, Swiftnote, Feathermark, Keynib, and Crosskey. The final name wasn't even my idea. The credit for it goes to Shaun Farrugia. But I liked it and it stuck.

Next came logo design. I've always had a certain love for RPGs and I liked the way the old timey bags full of gold looked. It's not a perfect fit for the app at hand, but the design came together quickly and I was quite pleased.

And finally the landing page. I grabbed myself a bit of Tailwind and Jekyll and got to work. For what its worth I still have no idea how marketing experts do it, but I discussed the copy with people, I polished the experience of reading it for the first time, I made sure it looks decent on all the devices I have around and then it got pushed to live.

Since going live it managed to convince three people to get the app, so I guess I'm not doing too bad!

Launching

And here we stand, shortly before the big launch. This is really the first time I have built a proper product of my own and I am excited and also terrified of where the journey goes next. Here's hoping that the name "Notebag" can eventually stand proudly among the big stars of note taking.

Look for the launch on ProductHunt on Monday, May 4! And in the meantime, feel free to check out Notebag at notebag.app!

...

Oh, this isn't the support group? Fine. But nevertheless, let me tell you about a humble command-line tool called wp (or WP CLI). It is by far my favorite way to manage WordPress installations and saves me from having to click through the Dashboard approximately 95% of the time.

Installation

Getting wp installed isn't much of a hassle. If you already have Composer installed it's a breeze. You need only require the package and you're done.

I usually have wp installed globally because I manage a lot of WordPress sites:

composer global require wp-cli/wp-cli

But of course you can also just require wp for a project locally:

composer require wp-cli/wp-cli --dev

If you don't have Composer installed, there's also some alternative installation methods. Feel free to pick the one that best suits your workflow.

✨ Congratulations! Your life just became a whole lot more efficient!

Installing Wordpress from the command-line

Now to see just what you can do with wp, let's install WordPress and configure it. All done from the cozy comfort of your shell. The very first thing we need to do is to download the latest version of WordPress.

$ wp core download

Downloading WordPress x.y.z (en_US)...
md5 hash verified: 83bec78836aabac08f769d50f1bffe5d
Success: WordPress downloaded.

That wasn't hard at all, was it? Now let's setup our wp-config.php with all the necessary information. For this you can run the following command

$ wp config create --dbname=wpdemo --dbuser=root --dbpass=root

Success: Generated 'wp-config.php' file.

Of course you should replace the above database information with your own. The next step is to actually run the WordPress installer. That looks a bit like this

$ wp core install \
          --url="https://wpdemo.test" \
          --title="WordPress Demo Site" \
          --admin_user="pretzelhands" \
          --admin_password="awesome-p4ssw0rd" \
          --admin_email="hello@pretzelhands.com"

Success: WordPress installed successfully.

And you're done. You can now visit the URL of your WordPress site and take a look at your wonderful new installation!

That's a lot of parameters to remember. But never fear, wp is here! Until you remember all these parameters you can also run wp config create --prompt and wp core install --prompt. These commands will interactively ask you for all required parameters. Nifty!

Now let's talk about some other awesome features you might want to have.

Installing plugins

Check! Grab the slug of a plugin and you can run the following command:

$ wp plugin install advanced-custom-fields

Installing Advanced Custom Fields (x.y.z)
Downloading installation package from https://downloads.wordpress.org/plugin/advanced-custom-fields.x.y.z.zip...
Unpacking the package...
Installing the plugin...
Plugin installed successfully.
Success: Installed 1 of 1 plugins.

$ wp plugin activate advanced-custom-fields

You can also pass in a --activate flag to activate the plugin immediately. This looks like

$ wp plugin install advanced-custom-fields --activate

If you don't know the slug of a plugin you can find it by going to the plugin repository, searching for your plugin and then checking what it says after the last slash.

For example the URL for the Advanced Custom Fields plugin looks like this: https://wordpress.org/plugins/advanced-custom-fields If you check everything after the last slash you can see the slug is advanced-custom-fields. This is what you pass to wp. It may take some time to remember what slug each plugin has, but you'll remember your favorites soon enough!

Installing languages

I hail from a German-speaking country. If I hand over a WordPress site with an English dashboard to a local client, they'll probably look at me weird. Thankfully, installing languages with wp is a breeze.

$ wp core language install de_DE --activate

Downloading translation from https://downloads.wordpress.org/translation/core/x.y.z/de_DE.zip...
Unpacking the update...
Installing the latest version...
Translation updated successfully.
Success: Language installed.
Success: Language activated.

And you're done! Go grab some tea or coffee and enjoy your success. The --activate flag here works the same way it does with plugins.

Installing a theme

By now I'm sure you've figured out the pattern. But here's how you install a theme and activate it

$ wp theme install storefront --activate

Installing Storefront (x.y.z)
Downloading installation package from https://downloads.wordpress.org/theme/storefront.x.y.z.zip...
Unpacking the package...
Installing the theme...
Theme installed successfully.
Activating 'storefront'...
Success: Switched to 'Storefront' theme.
Success: Installed 1 of 1 themes.

And now you have Storefront for WooCommerce installed and running. Yes, that's really all there is to it.

Closing notes and further reading

The three examples given above showed you the most common use cases for wp, but you can manage pretty much anything with it. Here's a short excerpt of what you can do without ever touching the WordPress dashboard

• Manage users
• Create, update, delete your posts
• Create, update, delete and moderate comments
• Do a search-replace in your database
• Export and import WordPress data
• Manage multi-site networks
• Scaffold code for custom post type, taxonomies, child themes, etc
• Run a custom PHP shell for your WordPress install
• Run a local development server for your Wordpress install
• ... (The list does go on for a while longer)

And if you install WooCommerce for example you get another set of commands related just to managing your online store.

So if you're working with WordPress, especially if you're working with more than 2-3 sites I heartily recommend you go out, install wp and get more free time out of your day.

If you want to read more, you can check out the documentation, it is an excellent reference. Otherwise, a lot can be found out by running wp help and browsing the man page.

Enjoy!~

Sometimes this is very much me when it comes to side projects. Don't get me wrong, I love building side projects and really want to get as many things out there as possible. But sometimes the rabbit holes of the internet can be crazy deep and what starts out as a quick check in the documentation can end up in hours of reading about some esoteric programming language you'll never make use of.

I admit that I might have a bit of a procrastination problem. It happens to the best of us.

But there's a nice way to fight it and share with other people at the same time: Live streaming while you code. This has been a thing for a while now, but especially since shipstreams.com by fellow Austrian programmer Armin Ulrich showed up, there's been a bit of a boom around it. And I joined in back when it first started.

Turns out streaming is a bit like pair programming on demand but with more of a social factor. Having someone watch you and help you when you're stuck is like a super power. At the same time you get to talk about what you're doing and interact with people who are actually interested in what you're doing feels both incredibly humbling and motivating.

Generally there's only like 4-5 people at any one time watching my stream, but knowing that they are spending their free time to see me code makes me a lot more focused and prevents me from accidentally wandering off on tangents of non-productive behavior. In the end it's a bit more exhausting than coding by yourself, but it also feels like I get at least twice as much done in the same time.

Also whenever you find an error you will be doubly invested to fix it and to keep things moving along. Yes, your code might not always be super pretty while you're live, but at least you get stuff done! And sometimes that's the important thing. You can fix code that has been written, but you can't do anything with code that does not exist.

So if you ever find yourself in a creative rut, give streaming a shot! Set up an account on Twitch and Shipstreams, learn how to set up your PC/laptop for streaming and off you go! It doesn't have to be perfect, it doesn't have to be ultra professional. People will be happy enough to watch you either way. It's great fun and maybe you can learn a thing or two.

And for a shameless plug: You can follow me on Twitch to see when I go live. I'd love to chat with all of you!

But sometimes, in your personal projects, you should try throwing the rules out of the window. Just chuck them out and play around! One such example is the code behind my blog. It's written in PHP without a framework of any kind. It's a loose collection of loose PHP files with a bunch of rewrite rules holding it together. It's basically like a non-polished version of Jekyll.

Here's what the folder structure looks like using tree

blog
├── bootstrap.php
├── composer.json
├── composer.lock
├── lib
│   └── Post.php
├── ps
│   ├── 2019-01-02-coloring-terminal-text.md
│   ├── 2019-01-03-non-obvious-behaviors.md
│   ├── 2019-01-04-command-line-flags.md
│   ├── 2019-01-05-bringing-your-entire-infrastructure-down.md
│   ├── 2019-01-07-jinx.md
│   ├── 2019-01-09-how-i-got-into-freelancing.md
│   ├── 2019-01-12-mentoring-junior-team-members.md
│   ├── 2019-01-14-building-responsive-email-with-mjml.md
│   └── 2019-01-15-breaking-the-rules.md
├── public
│   ├── assets/ (images etc.)
│   ├── blog.php
│   └── index.php
├── vendor/ (dependencies)
└── views
    ├── base.mustache
    ├── blog.mustache
    ├── listing.mustache
    └── partials
        ├── about-me.mustache
        ├── description.mustache
        ├── header.mustache
        ├── newsletter-form.mustache
        └── title.mustache

The public folder is what gets exposed. The ps folder contains all of my posts written in Markdown with frontmatter. The lib folder holds the class responsible for loading posts.

Internally a URL like https://pretzelhands.com/posts/jinx gets rewritten to https://pretzelhands.com/blog .php?slug=jinx. I then search the files to match the correct one based on the slug. I parse the post date from the title. All of that happens in a Post class. Before that the code was directly in blog.php along with some nasty mixed up HTML.

Here's what that looked like

Is it ugly code? Unbelievably so. Was it fun to write? You bet! The entire blogging system was running in an hour or two on a Sunday before going back to work. I didn't want it to be pretty, I just wanted to get it out there. And whenever some code becomes unbearable to look at some selective refactoring happens.

And I love hacking on this thing. I bolt on some code, upload it to my server and call it a day. There's some globals running wild, there's code duplication, there's all sorts of stuff I would get yelled at for during a code review. But no one will ever see the code for my little blog, so I'm totally okay with doing this. It's my own messy little playground. I keep adding to it and see what sticks.

So if you're ever working on a small project of your own that isn't intended to be The Next Big Thing™, why don't you just go all out and write code as if you're throwing color at the wall? It's incredibly therapeutic, I tell you!

Here's an example from a project didn't see the light of day

require_once('../../server/bootstrap.php');

use Respect\Validation\Validator;
use Carbon\Carbon;

$database = getDatabaseConnection();
$emailValidator = Validator::email();

if (!isset($request->email) || !$emailValidator->validate($request->email)) {
    sendResponse([
        'error' => true,
        'errors' => [
            'Please provide a valid email for logging in.'
        ],
        'errorFields' => [
            'email'
        ]
    ], 400);
}

There's no standardization to any of this. The sendResponse function was a little wrapper that just sent a few headers, serialized the array into JSON and echoed it all. $request is just the STDIN PHP provides wrapped in json_decode It's beautiful in its ugliness.

Disclaimer

This does not mean I think you should write production code as if you're some abstract artist and ignore all rules and conventions at work. Those conventions usually have good reasons behind them. But I feel like if you're programming for your own enjoyment it helps to be more creative and experimental. To get rid of rigid structures. Who knows, maybe you find something you really like and can refine and improve.

So don't go around and tell people that Richard gave you the permission to write spaghetti code. 😁

Enjoy!~

If you count yourself among those that have never built an HTML email before, count your blessings. The amount of work that has to go into these magical creations can be absolutely mind-numbing. HTML email is like the Wild West of web technologies.

I wish I was kidding about this, I really do. Building HTML email was one of my responsibilities at my first job before I got into freelancing and I feel like in those three years I aged by at least a decade. The amount of tiny, obscure things you have to remember is incredible. An affront to everyone even. We're talking inline styles, table layouts, transparency hacks, and so and so forth. And funnily enough Google, the company responsible for one of the most advanced browers, is one of the worst offenders.

No really, here's a list of CSS support in email. Be sure you sit well before reading it. It's a hoot and a half.

Of course for a while now there's been solutions like Foundation for Emails, emailframe.work and many others. But they're either terrible to set up or still require you to remember a good few pitfalls in HTML email. But a while ago, thanks to @pugson I learned about one framework that stood above all others.

It's called MJML and it rocks my socks.

It came a bit too late for my mainstay in email marketing, but I still use it for transactional emails. And this is your whirlwind guide to building HTML with it! So let's get started.

Installing and running MJML

Installing MJML is quite simple if you have npm or yarn installed. You need only run the following command and you're good to go.

npm install -g mjml

If you're not part of the web crowd and don't have MJML or if you just want to try it out and save some time, you can also use their online playground. I admit that I mostly use that one, simply because it's super convenient to see your email right next to your code.

To learn how to use the command-line tool in detail, be sure to check out the excellent documentation.

But as quick summary here are the commands you will most likely want to use

# Compile the MJML file and save it to an HTML file
mjml email.mjml -o email.html

# Watch an MJML file for changes and recompile automatically
mjml --watch email.mjml -o email.html

# Minify the compiled HTML to save space
mjml --config.minify email.mjml -o email.html

Building your first email

As an example email to build I will pick one made by yours truly. It's a magic link email I'm using for a meeting tracker app I'm currently building.

It's not particularly complex, so I think it's a perfect fit for a first go at MJML. First, let's take a look at the entire template as it stands and then let's take it apart bit by bit.

<mjml>
    <mj-head>
        <mj-font name="Montserrat" href="https://fonts.googleapis.com/css?family=Montserrat:400,700" />
        <mj-preview>Click the button inside to log into your account</mj-preview>

        <mj-attributes>
          <mj-text font-size="16px" line-height="24px" padding="0px 48px 16px" />
          <mj-button font-weight="bold" font-size="16px" background-color="#8e2de2" />
          <mj-divider border-color="#8e2de2" />
          <mj-all font-family="Montserrat, Helvetica, Arial, sans-serif" />
        </mj-attributes>

        <mj-style inline="inline">
          .link {
            color: #8e2de2;
            text-decoration: none;
          }
        </mj-style>
    </mj-head>
    <mj-body>
        <mj-section padding-bottom="0">
            <mj-column>
                <mj-image
                        padding-top="20px"
                        padding-bottom="20px"
                        width="200"
                        src="https://howmuchdoesthismeetingco.st/assets/logo-hmdtmc.png"
                />

                <mj-divider />

                <mj-text align="center">
                    <h1>Your magic link</h1>
                </mj-text>

                <mj-text>
                    Somebody requested a log in from your account. If that was you click the button
                    below to sign into your account.
                </mj-text>

                <mj-button href="https://howmuchdoesthismeetingco.st/token/abcdefg123456789">
                    Log in now
                </mj-button>

                <mj-text>
                    <br />
                    Have a wonderful day,<br />
                    <strong>Richard from HMDTMC</strong>
                </mj-text>

                <mj-divider />

                <mj-text
                        padding="8px 48px"
                        font-size="11px"
                        line-height="16px"
                        color="#999999"
                >
                    If the button up doesn't work, you can use this link:
                    <a
                            href="https://howmuchdoesthismeetingco.st/token/abcdefg123456789"
                            class="link"
                    >
                        https://howmuchdoesthismeetingco.st/token/abcdefg123456789
                    </a>
                    <br/>
                    Your login link expires after 1 hour. You can request another one at any time.
                </mj-text>
            </mj-column>
        </mj-section>
    </mj-body>
</mjml>

Okay, that was a lot of code and if you're not running yet, I'm proud of you. The most complex part is probably right at the beginning in mj-head, but let's start from the very beginning: the root element.

The document structure

MJML is not unlike HTML or XML. It is after all a markup language that tries to spit out HTML. As such, each MJML document begins with a simple root tag:

<mjml></mjml>

Inside this root element you have the same sections you're used to from a regular HTML document. One head and one body. They're just prefixed with mj. So the standard structure looks like so:

<mjml>
    <mj-head></mj-head>
    <mj-body></mj-body>
</mjml>

The mj-head is a pretty exciting place where you define all of your styling and meta data! It's probably the most fun part of writing an MJML-based email.

The document head

Now let's look at all the weird stuff in the mj-head and demystify it a bit.

<mj-head>
    <mj-font name="Montserrat" href="https://fonts.googleapis.com/css?family=Montserrat:400,700" />
    <mj-preview>Click the button inside to log into your account</mj-preview>

    <mj-attributes>
      <mj-text font-size="16px" line-height="24px" padding="0px 48px 16px" />
      <mj-button font-weight="bold" font-size="16px" background-color="#8e2de2" />
      <mj-divider border-color="#8e2de2" />
      <mj-all font-family="Montserrat, Helvetica, Arial, sans-serif" />
    </mj-attributes>

    <mj-style inline="inline">
      .link {
        color: #8e2de2;
        text-decoration: none
      }
    </mj-style>
</mj-head>

As the very first child of our document head we have a mj-font tag. It takes in a name and an href. If you look at where the href links to, you'll quickly figure out that all this does is load in a font hosted on Google Fonts. Of course, given the state of affairs, not all email clients support this. But some do and they get rewarded with prettier typography.

<mj-font name="Montserrat" href="https://fonts.googleapis.com/css?family=Montserrat:400,700" />

After this we find an mj-preview tag that contains some text. This is an invisible preview text that gets displayed in the inbox of some email clients right below the subject of the email. To give you some context, I'm talking about the light-gray text depicted here (in Apple Mail)

You can use this to tease your subscribers about the content of your newsletter or give a summary of the contents. It's a bit of an art to find text that is just long enough to fill out the entire area, but not too long to be cut off by the email client.

<mj-preview>Click the button inside to log into your account</mj-preview>

Next up is a special one. mj-attributes allows you to set default styling for all MJML tags that you're using. You can of course override these at any time. These just exist so you don't have to repeat yourself over and over again.

Each child of the mj-attributes tag is another MJML tag such as mjml-text or mjml-button. Each of the CSS properties you want to use is defined as an attribute. This may be unusual at first, but you'll get used to it very quickly. There's also a special mj-all tag, which can be used to apply a style to all elements. (A bit like the CSS * selector)

Also note how I define some fallback fonts for the mj-all tag. This is so that email clients with bad CSS support still have at least a somewhat bearable font to look at.

<mj-attributes>
    <mj-all font-family="Montserrat, Helvetica, Arial, sans-serif" />
    <mj-text 
        font-size="16px"
        line-height="24px"
        padding="0px 48px 16px" 
    />
</mj-attributes>

And we wrap up our head component with an mj-style tag. You can use these to write regular CSS and media queries. One special thing to note here is that I added the inline="inline" attribute to my style tag. This makes MJML search for all elements with a class of link and adds a style attribute to it.

<!-- So this element -->
<a href="#" class="link">My awesome link!</a>

<!-- Turns into this -->
<a href="#" class="link" style="color: #8e2de2; text-decoration: none;">My awesome link!</a>

The reason why you would want this is once again rooted in the lack of CSS support in email clients. Many only support only basic selectors like a or h1, while Gmail just takes your style tag and throws it out entirely. Yup. That's actually a real thing that happens. Thanks Google.

The document body

This is where the meat of your email lives. MJML provides many custom tags to make building email as painless as can be. You can find a list of all tags available in the documentation.

I will highlight the ones I used below, starting with mj-section.

mj-section is meant to be used as a row in your layout. Imagine it a bit like Bootstraps or Foundations grid row elements. It offers some vertical spacing by default, but you can of course remove it. If you don't need any special thing like that you can of course also just use a div or table (and yes, I'm serious about the table).

mj-column allows you to define one or more columns in your layout. This way you can easily build a grid layout in your email. To have multiple columns next to each other put them into an mj-section and you're good to go.

An example of a 2x2 grid could look something like this

<mj-body>
    <mj-section>
        <mj-column>Row 1, Column 1</mj-column>
        <mj-column>Row 1, Column 2</mj-column>
    </mj-section>

    <mj-section>
        <mj-column>Row 2, Column 1</mj-column>
        <mj-column>Row 2, Column 2</mj-column>
    </mj-section>
</mj-body>

mj-image is an image. You probably knew that. By default MJML adds some styling to it to make it stand out a bit more, but once again, you can easily change that.

mj-text is a generic text container. You can put any p, h1 or whatever else in there and make it look pretty. If it's just a random blob of text you can also simply put that text in directly as a child.

mj-button is essentially a link that's styled like a button. Pass in href as an attribute and you've got a great call-to-action in less than 30 seconds. If that's not speedy email development I don't know what is.

mj-divider is a bit like an hr, except it's a lot easier to style. In my email template I used it to separate header and footer from the main content.

Styling tags

For each of the tags I named above, you can easily pass in attributes to give them a one-off style that you didn't add in your mj-attributes. For example I used this tag to style my footer text

<mj-text
    padding="8px 48px"
    font-size="11px"
    line-height="16px"
    color="#999999"
>
    Footer text
</mj-text>

If you end up using a set of styles more than once, but it is still not the default style you want, you can also define an mj-class that allows you to pass styles around like Oprah passes around cars.

You need only define the following in your mj-attributes

<mj-attributes>
    <mj-class name="padded" padding="20px" />
</mj-attributes>

Give you class a name and some CSS attributes you want it to use and off you go. Note however that this only works for MJML tags. For regular HTML tags you should make use of mj-style.

Summary

MJML is a great tool that helps you build responsive HTML email much faster than you would otherwise be able to. The predefined components help you get the most annoying parts out of the way so you can focus on the content and layout, instead of fiddling around with CSS hacks.

Of course, no tool is perfect and neither is MJML. So in the end, always be sure to test your email templates with a tool such as Litmus or EmailOnAcid. Just to be sure ;-)

Enjoy!~

On the other hand, I've recently found myself doing a lot of mentoring during my hours at $BIGCO that I mentioned in my previous post about freelancing and the people I've been speaking with seemed to be into it.

Things I write here may be complete bollocks, but hey, read the first paragraph again and you'll understand why. I guess this is partly just for me to think things through.

Be patient with everyone

I don't know if you've ever noticed this with other members in your team, but many people seem oddly annoyed when approached with questions by someone. It may be because they dislike being interrupted by others during work, it may be because they don't like being bothered with someone else's problems. But it's always a bit sad to see when someone has a question and the person they ask rolls their eyes or sighs as they get up from their table.

Hence I always, always, always try to be patient. I've had many people come to me for a question that took me all of a minute to show them the answer to. They got unstuck and could happily continue their work and I didn't lose any critical amount of productivity in mine. If a question in your team pops up a lot, write it down on an FAQ page.

And if you ever find yourself losing your train of thought, go and read this post by Swizec. No one will hold it against you if you take a few seconds to write down stuff.

Give them tools to solve the problem

One thing that can be nice to do is to not point somebody directly to a solution unless it's critical to your deadline. But if you have leeway in your project try and show them how they can find a solution themselves. They don't know how to get started with checking where things go wrong in their program? Teach them how to use breakpoints to step through their program! They can't figure out why their flexbox is behaving weirdly? Show them a site like flexbox.help let them describe their goal to you and let them click through it while explaining it to you.

If they really ran into something bigger they'll come back and ask for help again, but if they can figure it out themselves that feels like a huge win that they had independently of you. It gives a sense of confidence and independence.

So show them how to get there, but don't just point at the screen and tell them, they should have, obviously used justify-content: space-between instead of justify-content: space-around. Let them have their own wins.

Pair programming can help a ton

This is of course dependent of the type of person you're dealing with, but sometimes of people feel a lot more confident when pair programming. At times it already helps if you're the duck for their rubber duck debugging. While they speak out loud and explain the code to you, often times things will click in their head.

So if they're open to it and you have a few minutes, sit with them. Let them explain their code, walk through it with them. Sometimes the thought of someone having your back is all that is needed.

Listen to the individual

Of course, all of this varies from person to person. Junior devs that joined your team a week ago will most likely need more hand-holding than one who joined two months ago. Some people don't like somebody looking over their shoulder while they code, and that's fine too. For what it's worth I'll fumble through a docker run command if you just stand their and watch me.

Each person is different and each one requires a different approach to mentoring. But these are some things I found useful in helping others learn. Maybe you can find some use for yourself.

And above all, when a question seems trivial to you but insurmountable to your junior always remember: Once upon a time you were new to programming as well. ;)

Enjoy!~

Today's blog entry was decided on by Twitter, so for the people who are interested, here's the story of how I got into freelancing from the beginning all the way up to where we are now.

Since there may be a few people reading who are coming here for the first time, I'll give you some basic introduction of who I am, what my experience level is etc. Here goes nothing.

I'm Richard, currently 21. I started messing with HTML and CSS around October 2011 by sheer accident. Somewhere along the way I picked up JavaScript and I started messing with PHP in early 2013.

For all intents I have no real educational background. I dropped out of high school in sophomore year (2013) and decided to pick up a vocational training as media designer instead. As part of that I visited vocational school which is four grades consisting of 3x9 and 1x4 weeks of schooling. That's the absolute peak of my education. My highest achievement along traditional academia is a middle school graduation.

No, I'm not kidding. I've never learned about sine and cosine or anything like that. I know that if I enter a sine function in my calculator I get an even wave form. No idea what that's even useful for.

As of writing this post I'm entering my third year of business. Last year I finished with approximately $70,000 in gross revenue. Projections for this year are higher. Now, enough braggy stuff, let's get onto the meat of the story.

Q4 2016: Finishing my vocational training and moving across the country

This is where our story starts. In October 2016 I was about two months from the final exam for my vocational training. By then I had made friends with various colleagues and one in particular. There won't be any names for privacy reasons, but she suggested to team up and try going freelance. She would do the marketing and I would do the coding.

So that's what we did. Around the middle of October we both had an appointment at the Chamber of Commerce and set up our companies along with a joint company named Marivol. The name was based on mariposa for butterfly and volare for to fly.

At the same time I was about to move cross country from my home in Salzburg to the lovely little province of Burgenland. The combination of this move, my general being fed up with the company I previously worked at and starting a new company led me to one decision

"I will quit my job and go freelance full-time"

In hindsight that was probably not the smartest choice but it is what it is. So for the first two months while I was wrapping up my job situation, my colleague started looking for clients. We put up a website, printed flyers, the whole nine yards.

Q1 2017: Big trouble in little Burgenland

By the time 2017 rolled around we had pulled in a total of four clients. My colleagues idea was to start a bit lower with pricing due to lack of references. That sounded alright and once everything was settled and the dust cleared we had our first revenue.

It came out to approximately $1,500. Yup.

Divide that by two and you'll quickly realize that that's not a lot to live off of. I had savings of course and this was the moment where I was glad I had them. So while my partner continued the search I clenched my teeth and started building.

We finished projects for two of the clients. One client sent us a pre-payment and never replied to a design template we sent them. One client never managed to send us all the necessary data to set up her web shop. We were truly off to a great start. Eventually January became February became March. I turned 20 and had made a total of $400 from freelancing, when a bit of shocking news hit. My colleague couldn't sustain the effort next to her full-time job.

I felt a bit of a bit in my stomach and accepted the fact as it was. The very next day I did the best thing I could think of: Sign up for UpWork and find me some work. So I set up a profile and got to work.

Turns out that working on Upwork is hard. I mean real hard. I tried to play the numbers game and sent applications for jobs big and small and I did find some jobs using PHP and JavaScript. I even applied myself and got certified with their talent program. I added all the things to my profile, I did the tests and scored in the top 10% for as many as I could.

And while it wasn't an overwhelming amount, at least some money started rolling in eventually. By end of March I was up to approximately 400$/month from freelancing with my first repeat customers. It wasn't huge, but at least it slowed the drain on my savings which were growing smaller each month. But the worst was yet to come.

Q2 2017: Living in the Philippines and going broke

Eventually April 2017 rolled around and I went ahead with a plan I'd had for a bit more than a year by then. I was going to visit and live with my girlfriend in the Philippines for three months. This had a few useful benefits:

• Compared to Austria, staying in Manila is quite cheap
• I didn't have to pay household contribution to my parents
• I was with my girlfriend (Duh!)
• Also the food was good. I love isaw to this day.

So on April 15 I flew over and I was supposed to stay until July 15. During that time I kept on searching for scraps and bits on Upwork and all other sites I could find. I posted on Twitter about much of anything I did, simply as a publicity tool. But things didn't exactly get much better. I still hovered around 200-400$ of income per month, which was simply not enough to cover my living costs.

First the evenings out were cut down. Then the weekend trips. Eventually me and my girlfriend shared a single plate of food at a restaurant in her university to make the money last longer. I was still happy being there, but my time was noticeably running out.

Eventually me and my girlfriend hopped on a bus and travelled to some places in Northern Luzon, we stayed at the cheapest hotels possible and spent our evenings eating cheap takeout from 7/11 and watching Gilmore Girls. Two weeks later we returned to Manila when finally the decision was made that perhaps I should rebook my flight to save whatever I had left and return to home earlier. And so I did.

By this time I had someone reach out to me on Twitter. He asked if I would be willing to help him develop a few things. Apparently he had followed me since way back when I still dabbled in game development and had my old job. We were scheduled to call for a first time soon after I left the country.

So I ended my little flirt with Digital Nomadism and returned to Austria on July 1. All I had to my name was a lead for a freelance job and $100. This was the point at which I decided:

"Either this lead works out and keeps me afloat or I'm stopping this."

Q3-Q4 2017: The Pursuit Of Happiness

I arrived back at home pretty much hopeless and desperate. I took about a week off to replenish some energy and gather some confidence and the I went straight back to work. It turned out to be my last week-long vacation for approximately 1.5 years.

And I had a call. And this call went well. It went very well. So I started helping the person who had contacted me with developing a shop. I helped with developing a fishing trip portal.

On the side I was doing some work for previous customers. By the end of July 2017 I had somehow managed to scrape together an income of approximately $1,500. It wasn't a luxurious living, but for the first time in almost a year I had a living.

I paid back my parents for the leeway they gave me and pursued the lead further. It managed to come out to somewhere around $1,000 to $1,250 per month for this one lead, plus some extra money from other leads. I worked quite a lot during that time and developed a pattern of work schedule.

I'd get up around 8am and would work until 1pm for my big lead. I would then take off some time in the afternoon until maybe 4-5pm and enjoy the sun outside, try and get some of that Vitamin D. I would then return to scheduled calls and keep on coding until 12am or maybe 1am for other projects. It sounds crazy in hindsight, but damn, that was a happy time for me. It felt like I finally had achieved something.

And I don't know why, but once I had that first income rolling, other leads started popping up. A former classmate of mine worked at a local radio station allowing anyone to broadcast their own program. They wanted to relaunch their site from Typo3 to Wordpress and needed some 800 sites of content migrated. So I took the job. It took me two months next to my other jobs, but I had tried to high-ball my hourly rate at that point (which was maybe $45/h).

I got almost $5,000 for this project

This was huge to me! Somebody was actually willing to pay me proper money for my time. All said and done, after various deductions I ended my 2017 just barely below the line at which I have to start paying income tax. For reference, that is about $12,500 in Austria.

Q1-Q2 2018: Continued leads and increased income

So 2018 rolled around. I joined a group I have grown to love and cherish: wip.chat - The group inspired me and motivated me to do my all each and every day.

By this time thankfully there was just enough word of mouth to keep me going. I worked with my main leads from July 2017 and the local radio station along with some small-time maintenance tasks. All said and done it came out to some $2,000 to $2,500 per month. I was able to save again. I got to lease myself a new car.

At this point my old colleague from early 2017 also reached out. She had by now switched to working for the parent company for a few well-known brands and was doing digital marketing. Knowing that I was still around and programming she also generated a few leads for me that kept contributing to this roll I was own. I developed a little digital marketing asset platform for them.

In the spring my girlfriend came by to visit me in Austria for three months. And we had a good time this time around. There was no worry of being kicked out of anyone's house or my revenue stream suddenly draining to zero. Life was good. Also, I got engaged! So that was fun!

And then May 2018 rolled around. A recruiter contacted me on LinkedIn (I know, but hear me out). I responded politely and said that I would be happy to take on some freelance opportunities even for a longer period of time. Apparently they had had some trouble finding people to recruit since it was all very short-term.

I ended up driving to Graz and having an interview with the recruiter. I named him my hourly rate that was a good step above what I charged before (~$70) and he named me the amount of hours that would be approximately required of me each month. I agreed and drove home to do some maths. So I punched some numbers into the calculator and had a bit of jaw-dropping moment. All the thoughts in my head were:

"Wait a sec, they're not going to pay me $10,000 per month, right? I'm 21. That's ridiculous."

And then I was invited to an interview with that company. And they liked what I had to offer. And they hired me for a contract between June 2018 and December 2018. Yes. They paid me $10,000 a month.

Q3-Q4 2018: Holy shit, what is happening

So I started working at said company for payment integrations and I integrated into their team. I was spending approximately 30h/week in their office and then spent another 10-20h/week doing small maintenance tasks where required. Having a big name in my portfolio I upped my hourly rate a bit. I added the company to my LinkedIn profile.

The next 6 months passed rather quietly. I drove out to Graz to work and drove back home again. My office times being approximately 8.30am - 3pm followed by some commuting and some work in the evening. My average income varied between $12,000 and $14,000 each month (although those are gross revenue numbers). I just packed everything into a savings account and decided to live off of $1,500 per month at most.

And each month the money would arrive. And each month I was still shocked. So eventually November came around and the end of my contract drew near. I had a discussion with my superior at the company and after everything was said and done they offered to extend my contract for another 8 months, with more hours.

During this time I really didn't have much time nor energy for any side projects, but I did finally take a week of vacation in November. My first in a year and a half. And in December we signed the contract.

Q1 2019 and beyond

And now, now we're here. My contract runs until August 2019 and I have a month of leave in May due to my getting married lsdfjlsdf and moving my fiancée to Austria. My pay is still on a slight upward trajectory and I do not know where it will go next.

All I know is that right now I have leads approaching me from every side and that I don't really have the time and will to take them on. Starting with the new year I also found some more energy to invest in my side projects. In the first 9 days I've already started this blog, How Much Does This Meeting Cost? and a little nginx wrapper called jinx.

Things are good right now. Things have been good for a long while now. And I stand by one single statement: Throughout most of this journey I had little to no clue as to what I was doing. I lucked out. I really, really had an absurd amount of luck when I needed it the most. And sometimes, a little luck goes a long way.

If you find yourself having more questions, please do reach out to me on Twitter! Or maybe on Telegram!

Enjoy!~

I am super excited about today's blog post because I made a thing! Yes, I actually made a thing with my own two hands and now I'm releasing it! I present to you: jinx

👉 You can find it on GitHub. Go ahead and star it. I'll wait.

Done? Great. Now as you may remember, I very recently killed my entire server infrastructure by accidentally generating way too many Let's Encrypt accounts in a very short timeframe. It was super funny and I spent the better part of my Saturday getting everything back up on a backup nginx setup.

As part of that I quickly noticed a very repetitive pattern to setting up my virtual hosts. It went a little something like this for every site

# create a new site
cp \
    /etc/nginx/configurations/php.conf \
    /etc/nginx/sites-available/pretzelhands.com.conf

# change the host name to the new site name
nano /etc/nginx/sites-available/pretzelhands.com.conf  

# activate the site in nginx
ln -s \
    /etc/nginx/sites-available/pretzelhands.com.conf \
    /etc/nginx/sites-enabled

# restart to publish newly activated sites
systemctl restart nginx

Using ^R was a bit faster but still incredibly tedious. That's 4 different commands and I have to enter the same or a similar path a whopping four times. That just didn't feel good and so I remembered how much I wrote about shell-scripting recently and decided to make use of it to create jinx.

With it this same process now looks like this.

# create a new site and insert hostname in template file
jinx site create pretzelhands.com php

# activate the site and restart
jinx site activate pretzelhands.com -r

From four long, repetitive commands to two quite short ones. What a nice improvement. And it only cost me my Sunday afternoon.

The initial development effort may have been higher than setting the sites up manually but I'm still glad I did it because it was fun to do and I can reuse it all the time. Every time I create a new virtual host, this tool probably saves me some 2-3 minutes of repetitive typing.

There's also a few other features, but you can read up on everything in the documentation. I freely admit that this is probably my single most useful side-project to date. Also it got the approval of Pieter Levels so I guess that counts for something!

👉 Don't forget to star it on GitHub.

I hope you get as much joy and and practical use out of this tool as I do and I'll be trying to extend it with some more useful features such as activating HTTPS and building configurations from included snippets. We'll see how far I can take it!

Enjoy!~

Yesterday Caddy successfully crashed all websites running on my server thanks to automatic HTTPS certificates

Setting the stage

Yesterday I was moving my blog from blechi.at to pretzelhands.com - The reason being that the shared host that runs behind blechi.at (which is a domain I share with my dad), seems to block requests from various Asian countries. I don't know why it happens and I can't be assed to ask support about it.

Kill your infrastructure in 60 seconds

Caddy requests its certificates from Let's Encrypt, as everyone should be doing in $CURRENT_YEAR. So while I was happily rearranging my config for pretzelhands.com, I restarted my server multiple times. Apparently I had setup my service for Caddy wrong however, causing it to register another account with Let's Encrypt every time I restarted the server. Usually this isn't a problem, because I restart my server maybe once or twice. I was fiddling around a lot yesterday, however and kept on requesting more and more accounts.

Turns out that Let's Encrypt really don't like that.

Before I knew I hit a rate-limit and my entire server just came crashing down in a magical rainbow of red error messages. According to Let's Encrypt this rate limit for accounts applies to 10 accounts per 3 hours. I have been trying to wait the specified amount of time and request a fresh set of certificates. So far I haven't been successful. Maybe they're also pissy about me requesting multiple certificates for the same domain. I simply don't know.

So I did the next best thing and pulled up nginx, which was stilled installed on my server anyways. And now here we are.

How to prevent this

One thing I learned about very quickly yesterday is that Caddy offers a -ca flag with which you can point it to the staging environment of Let's Encrypt. That is what I should have done. I've already implented a little script and called it caddy-test. It will do just that.

I still love Caddy with all of my head, but this was a real shotgun to the foot moment. Woops. Now excuse me while I bring the rest of my infrastructure back up. Grmbl.

Let's take a look!

# arguments.sh

# Default values of arguments
SHOULD_INITIALIZE=0
CACHE_DIRECTORY="/etc/cache"
ROOT_DIRECTORY="/etc/projects"
OTHER_ARGUMENTS=()

# Loop through arguments and process them
for arg in "$@"
do
    case $arg in
        -i|--initialize)
        SHOULD_INITIALIZE=1
        shift # Remove --initialize from processing
        ;;
        -c=*|--cache=*)
        CACHE_DIRECTORY="${arg#*=}"
        shift # Remove --cache= from processing
        ;;
        -r|--root)
        ROOT_DIRECTORY="$2"
        shift # Remove argument name from processing
        shift # Remove argument value from processing
        ;;
        *)
        OTHER_ARGUMENTS+=("$1")
        shift # Remove generic argument from processing
        ;;
    esac
done

echo "# Should initialize: $SHOULD_INITIALIZE"
echo "# Cache directory: $CACHE_DIRECTORY"
echo "# Root directory: $ROOT_DIRECTORY"
echo "# Other arguments: ${OTHER_ARGUMENTS[*]}"

Phew. That looks like a whole bunch of code. It includes all process of catching command line arguments. But let's go through everything bit by bit. First, let's start with the default values.

Defining default values

# Default values of arguments
SHOULD_INITIALIZE=0
CACHE_DIRECTORY="/etc/cache"
ROOT_DIRECTORY="/etc/projects"
OTHER_ARGUMENTS=()

This is simple enough. If the user doesn't pass in a certain argument, we fill it with some default value we're happy with. Alternatively you can make the strings empty and check if these empty values are still there. In this way you can easily verify that you have all necessary arguments passed in. How you go about that is an implementation detail of your script and thus left as an exercise for the reader. I recommend tldp.org for learning about operators.

Style-wise I like defining my arguments in all-caps snake_case, because I generally treat them as constants that I do not modify. You may disagree and you're welcome to call them however you like.

Looping through arguments

for arg in "$@"
do
  .. SNIP ..
done

Looping through the arguments is equally simple. You simply loop over the magic $@ variable your shell provides to you. It contains an array of the exact command as it was called, starting after the file name.

So if you call your script using ./arguments.sh -i --cache=/var/cache --root /var/www/html/public my-project, then the array will look a bit like so

(
   $0 = ./arguments.sh
   $1 = -i
   $2 = --cache=/var/cache
   $3 = --root
   $4 = /var/www/html/public
   $5 = my-project
)

Note that the $@ variable does not contain the value of $0. If you however access $0 normally, it will return the file name you used to call the script.

For our purposes we loop over each entry in the array and put it in a temporary $arg variable. Now we can process the arguments.

Processing all arguments

The arguments will be processed in a switch-case statement. As you may have noticed in the full code sample above, those come with their own delightful idiosyncrasies in syntax. Like a lot of other things in shell scripting, really. A case statement looks like this:

    case $arg in
        .. SNIP ..
    esac

Now let's look at the various ways to process arguments and how to write switch cases.

Boolean flags

Boolean flags are those which may be there or not. A good example might be a --help flag. Parsing those looks like so

-i|--initialize)
SHOULD_INITIALIZE=1
shift # Remove --initialize from processing
;;

This case statement checks whether the current value of $arg is either -i or --initialize. In our case this is true and thus we set the SHOULD_INITIALIZE variable to 1 to indicate that the flag is present. Afterwards we pop the value $arg off of our $@ array using shift. It now looks like the following:

(
   $0 = ./arguments.sh
   $1 = --cache=/var/cache
   $2 = --root
   $3 = /var/www/html/public
   $4 = my-project
)

Equals-separated flags

Our next case statement parses command-line flag of the form --arg=value, which is the traditional style of passing arguments. You can often see this when using Unix tools such as ls --color=auto.

-c=*|--cache=*)
CACHE_DIRECTORY="${arg#*=}"
shift # Remove --cache= from processing
;;

In this case we check if the current $arg matches the either -c= or --cache= followed by any number of characters. If it does we take that arg variable into our string and remove the parts of it we don't need. The #*= part looks super confusing at first. What it does is remove everything character from the beginning of $arg until it finds an equals sign.

This means that --cache=/var/cache becomes /var/cache. If you want to read up more on the topic of parameter substitution in shell scripts, I recommend this article from cyberciti.biz

After this our $@ array of arguments now looks as follows:

(
   $0 = ./arguments.sh
   $1 = --root
   $2 = /var/www/html/public
   $3 = my-project
)

Space-separated flags

Our third case statement handles command-line flags of the form --arg value, which is a more modern approach. You can usually see it with command-line tools written with Node.js or Python.

-r|--root)
ROOT_DIRECTORY="$2"
shift # Remove argument name from processing
shift # Remove argument value from processing
;;

Compared to the previous handler, this one is again rather easy to understand. We check whether $arg is equal to -r or root then we take the value of $2 into our ROOT_DIRECTORY variable and shift twice.

Why do we take $2? Remember: We have shifted away all previous arguments passed to the script so that now $1 is equal to the value of $arg and thus $2 now contains the arguments value.

After we shift the next two values off, we remain with this arguments array

(
   $0 = ./arguments.sh
   $1 = my-project
)

As the last step we will handle all the other arguments passed in without a flag. Let's go!

Matching other arguments

Our final case matches any value that wasn't matched by our previous handlers. These can be arguments passed without any flag, like a project name, or something else entirely.

*)
OTHER_ARGUMENTS+=("$1")
shift # Remove generic argument from processing
;;

For this handler we simply take the value of $1 and add it to a miscellaneous array. After all the additional arguments have been added to the array, you can decide to do whatever you like. For example the first entry in the array could be a project name. Who knows!

Trying it all out

Now if you add some echo statements and try to run your script as stated above with ./arguments.sh -i --cache=/var/cache --root /var/www/html/public my-project you could see output like the following

$ ./arguments.sh -i --cache=/var/cache --root /var/www/html/public my-project

# Should initialize: 1
# Cache directory: /var/cache
# Root directory: /var/www/html/public
# Other arguments: my-project

Closing thoughts

I think that the use of such a switch-case statement together with some more advanced features of shell scripting makes for a really nice and extendable way to add command-line arguments and flags to your scripts. It also allows for great flexibility, so if you don't like being stuck with one style you can easily use the other.

Enjoy!~

Escaping

When using a variable, you will always want to use double quotes when getting it's value. This is because otherwise your terminal of choice might decide to interpret a number of characters in it's own, delightfully destructive way.

An easy example can be given with looping over a variable's contents without using quotes.

list="one two three"

for word in $list
do
    echo "$word"
done

# Contrary to what you might expect this will output
# three variables in sequence:

## => one
## => two
## => three

Woops! Looks like your terminal just turned your nice string into a whitespace-delimited array. Which is clearly what you wanted all along, no?

To prevent this, you need only wrap the $list variable in quotes, like so

list="one two three"

for word in "$list"
do
    echo "$word"
done

# This will now output the expected string

## => one two three

So as an easy rule to remember: Unless you're on a treasure hunt for pain, just always wrap your variables in quotes.

Conditionals

Conditionals in shell scripting is super fun, because wrapping things in various bracket formats is totally what you expect to do. But those brackets hold deep dark mysteries that can easily trip you up when you want to do simple conditionals.

Let's do a simple number comparison. For our intents and purposes let's agree that 10 is bigger than 5. Now consider the following example

# conditionals.sh

ten=10
five=5

if [ "$ten" > "$five" ]
then
        echo "ten is bigger than five"
else
        echo "ten is smaller than five"
fi

The outcome should be quite obvious, no? Run the script and we should see ten is bigger than five. Easy!

And that's where you're wrong. Here's what the actual output is.

./conditionals.sh: line 6: 10: No such file or directory
ten is smaller than five

Wait, what? We're getting a file not found error and according to the shell 10 is smaller than 5. I don't know what math class you took, but in my world that's just all kinds of wrong.

What happened?

The first thing that went wrong is that we used a single pair of brackets. When we do that, the shell will automatically try to execute and compare to a file glob pattern and trips up because it can't find a directory named >. The easy way to fix this is either to use double braces like so: [[ "$ten" > "$five" ]] or just escape the greater-than symbol with a backward slash like so \>.

This will get rid of the first error, however you will still receive output telling you that ten is smaller than five. The problem here is that by using square brackets, the shell will automatically do a lexical comparison. And in the alphabet one and zero generally come before five.

Now there's two ways we can correct this, and I'll show you both.

# conditionals.sh

ten=10
five=5

# Solution 1
# This version uses the -gt switch to force
# arithmetic comparison.
if [ "$ten" -gt "$five" ]
then
        echo "ten is bigger than five"
else
        echo "ten is smaller than five"
fi

# Solution 2
# This version uses a pair of parentheses (( )) to force 
# arithmetic comparison.
if (( "$ten" > "$five" ))
then
        echo "ten is bigger than five"
else
        echo "ten is smaller than five"
fi

There are lots of other different comparison operators like -gt and you can find an extensive guide over at tldp.org along with the exact comparison syntax.

Functions

For the last chapter of this wild joy ride through shell behavior, we will take a look at functions. They too behave in ways that might be confusing if you come from other programming languages.

Calling a function

Contrary to basically every other programming language out there (save for some exceptions), bash does not use parentheses for a function call. You simply write the name. If you want to pass in some arguments, just add them with a space after the function call. It looks like this:

function foobar {
    echo "Foo to the bar"
}

foobar
foobar "argument1" "argument2"

Both calls will do the exact same thing. One thing to note is that you can pass any number of arguments to a shell function and it will not care. If you don't use arguments, they will simply be ignored. Another thing to be aware of is that you do not need to write parentheses after the function name. Your terminal simply doesn't care.

Using arguments

Unlike other programming languages, arguments in shell scripting are not defined in the function signature, but are rather used by using special $ variables, like so.

function foobar {
        argument_1=$1
        argument_2=$2
        argument_list=$@

        echo "First argument: $argument_1"
        echo "Second argument $argument_2"
        echo "List of arguments: $argument_list"
}

foobar "argument1" "argument2"

# Generated output
## => First argument: argument1
## => Second argument argument2
## => List of arguments: argument1 argument2

This also means that it's completely up to you to check whether an argument is actually set and throw an error if that is not the case. If you want to output nice error messages I highly recommend my post on coloring your terminal output. Ahem.

Variable scope

Another fun thing about shell scripts. If you define variables in your function like I did above, they are by default in global scope and can be accessed by anyone after the first time the function was called. Check this out

function foobar {
        argument_1=$1
        argument_2=$2
        argument_list=$@

        echo "First argument: $argument_1"
        echo "Second argument $argument_2"
        echo "List of arguments: $argument_list"
}

foobar "argument1" "argument2"
echo "Outside the function: $argument_1"

# Generated output
## => First argument: argument1
## => Second argument argument2
## => List of arguments: argument1 argument2
## => Outside the function: argument1

To prevent this behavior, you simply need to prefix your variables with the local keyword to prevent the function scope from spilling into the outside world. It looks like so

function foobar {
        local argument_1=$1
        local argument_2=$2
        local argument_list=$@

        echo "First argument: $argument_1"
        echo "Second argument $argument_2"
        echo "List of arguments: $argument_list"
}

foobar "argument1" "argument2"
echo "Outside the function: $argument_1"

# Generated output
## => First argument: argument1
## => Second argument argument2
## => List of arguments: argument1 argument2
## => Outside the function: <nothing>

And as grand finale, let's talk about return values!

Returning values

You'd think that using a simple return statement would be enough, yes? I hate to break it to you, but no. No, of course not. Because doing so would be entirely too easy. Instead what you do is to echo your return value and then capture the output into a new variable. This looks somewhat like this

function foobar {
        local argument_1=$1
        local argument_2=$2
        local argument_3=$3

        echo "$argument_1 $argument_2 $argument_3"
}

foobar_returned=$(foobar "foo" "bar" "baz")
echo "Foobar return value: $foobar_returned"

# Generated output
## => Foobar return value: foo bar baz

Phew. Makes you feel like somebody repairing their boat by putting duct tape over all the leaky holes.

Final thoughts

As you can see the world of shell scripting has lots of traps for unsuspecting people strolling past. I hope this post helps you avoid some of these traps in your own scripts. For a concise reference on most shell syntax you can check out learnxinyminutes.com. It's my go-to reference for odd quirks and things I just cannot seem to remember.

Enjoy, and good luck out there!~

Now, I remembered of course that there's possibilities to color your text in the terminal and npm for example has a wonderful package called chalk to do this. But in the environment I'm dealing with there is no npm and frankly Bash just feels like a more direct way of writing automation scripts, so I figured that there must be an easy way to do this.

And of course there is. All you have to do is use some escape sequences and a flag when using echo. And that's really all there is to it. First, let's define the available color codes as variables.

COLOR_RED="\033[0;31m"
COLOR_RED_LIGHT="\033[1;31m"

COLOR_GREEN="\033[0;32m"
COLOR_GREEN_LIGHT="\033[1;32m"

COLOR_ORANGE="\033[0;33m"
COLOR_YELLOW="\033[1;33m"

COLOR_BLUE="\033[0;34m"
COLOR_BLUE_LIGHT="\033[1;34m"

COLOR_PURPLE="\033[0;35m"
COLOR_PURPLE_LIGHT="\033[1;35m"

COLOR_CYAN="\033[0;36m"
COLOR_CYAN_LIGHT="\033[1;36m"

COLOR_GRAY="\033[1;30m"
COLOR_GRAY_LIGHT="\033[0;37m"

COLOR_BLACK="\033[0;30m"
COLOR_WHITE="\033[1;37m"

COLOR_END="\033[0m"

These are the 16 most commonly supported colors for various terminal emulators. Feel free to mix and match as you require them. The reason why we define these as variables is so that we can use them for formatting more easily. I won't stop you from remembering all the escape sequences, though!

Now that we have all the color codes defined we can do something like the command below.

echo -e "
    ${COLOR_GREEN}Success!${COLOR_END} 
    The task was executed successfully!
"

The things to note here are as follows: First, whenever you want to style your string with colors, you must add the -e flag to echo, otherwise you'll just end up with a garbled mess in your terminal. Second, whenever you are finished with using a color be sure to close it with a ${COLOR_END} tag. Otherwise the color will keep going on and on.

There are many more formatting options available in the terminal. To get an overview of all of them I recommend you visit FLOZz' wiki page on the subject.

Enjoy!