The first step was to install the Terraform tool (I am using Debian), so I followed their tutorial: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli that worked great, no issues.

Next, I need something simple to prove that something is working, so why not create a simple Hello World project (or copy and paste from somewhere?)

Let’s create a main.tf file with contents:

resource "null_resource" "default" {
  provisioner "local-exec" {
    command = "echo 'Hello World'"
  }
}

Then, to start this project, you need to run the following:

terraform init

It downloads if something is needed.

To run this plan, you can run a command to see the terraform plan (what it intends to do):

terraform plan

And when it looks ok, you can run the command (it will ask you to write yes):

terraform apply

hello-world$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + Create

Terraform will perform the following actions:

  # null_resource.default will be created
  + resource "null_resource" "default" {
      + id = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes.

null_resource.default: Creating...
null_resource.default: Provisioning with 'local-exec'...
null_resource.default (local-exec): Executing: ["/bin/sh" "-c" "echo 'Hello World'"]
null_resource.default (local-exec): Hello World
null_resource.default: Creation complete after 0s [id=589244851261473893]

Good, at this point, that was simple. But boy, I spent days trying to build a working Debian VM using libvirt.

First issue:

I wanted Terraform to manage the remote libvirt server via SSH. All examples didn’t work; they complained that the SSH server wasn’t in the known host list (I tried adding known hosts and known_hosts, but that just refused to work). So, I went this route:

  uri = "qemu+ssh://markom@homelab.lt:9022/system?known_hosts_verify=ignore"

Second issue:

There was a weird permission issue that looked like Terraform couldn’t manage VM images; I tried multiple folders and setting permissions to 777, but nothing worked. Then I found this issue: https://github.com/dmacvicar/terraform-provider-libvirt/issues/546

So, just changing this file /etc/apparmor.d/libvirt/TEMPLATE.qemu it worked:

#
# This profile is for the domain whose UUID matches this file.
#

#include <tunables/global>

profile LIBVIRT_TEMPLATE flags=(attach_disconnected) {
  #include <abstractions/libvirt-qemu>
  /var/lib/libvirt/images/**.qcow2 rwk, /var/lib/libvirt/images/**.raw rwk, /var/lib/libvirt/images/**.img rwk,
}

Third issue:

network_config and user_data can’t be set at the same time, so I decided for now go with default network settings (holy cow, I spent too long on this)

Fourth issue:

For some reason, qemu-guest-agent after installing doesn’t start, so I needed to add this line to cloud_init.yml:

runcmd:
  - [ systemctl, start, qemu-guest-agent ]

Fifth issue:

The issue is described here: https://github.com/hashicorp/terraform/issues/3287

Final result:

https://github.com/Markomas/terraform_debian

Docker setup

Well, because I want pulsar with the manager, I used a sample docker-compose.yml file:

version: "3.7"
services:
  pulsar:
    image: apachepulsar/pulsar:2.6.0
    command: bin/pulsar standalone
    hostname: pulsar
    ports:
      - "8081:8080"
      - "6650:6650"
    restart: unless-stopped
    volumes:
      - "./data/:/pulsar/data"
  dashboard:
    image: apachepulsar/pulsar-manager:v0.2.0
    ports:
      - "9527:9527"
      - "7750:7750"
    depends_on:
      - pulsar
    links:
      - pulsar
    environment:
      SPRING_CONFIGURATION_FILE: /pulsar-manager/pulsar-manager/application.properties

Just run docker-compose up

Then, on the local box run this command to create a user and password:

    CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)
    curl \
       -H 'X-XSRF-TOKEN: $CSRF_TOKEN' \
       -H 'Cookie: XSRF-TOKEN=$CSRF_TOKEN;' \
       -H "Content-Type: application/json" \
       -X PUT http://localhost:7750/pulsar-manager/users/superuser \
       -d '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "username@test.org"}'

Then in the browser just go to: http://localhost:9527/#/environments And we are greeted with the login page:

User: admin Password: apachepulsar

In this window, we can add a new environment (Pulsar Broker):

It seems to be working… Useless, but working… Using CLI commands, you can create topics/messages/subscribe to all good topics. The next things on the todo list: Pulsar SQL, tiered storage, transactions, adding security… But I will explore it further in the next posts.

So, first thing first: I have no idea how DHT works 😞

From a quick Google search, I see that there is no working example of this in PHP 😞 😞 😞

But I found this:

    import bencode
    import random
    import socket
    
    
    # Generate a 160-bit (20-byte) random node ID.
    my_id = ''.join([chr(random.randint(0, 255)) for _ in range(20)])
    
    # Create ping query and bencode it.
    # "'y': 'q'" is for "query".
    # "'t': '0f'" is a transaction ID which will be echoed in the response.
    # "'q': 'ping'" is a query type.
    # "'a': {'id': my_id}" is arguments. In this case there is only one argument -
    # our node ID.
    ping_query = {'y': 'q',
                  't': '0f',
                  'q': 'ping',
                  'a': {'id': my_id}}
    ping_query_bencoded = bencode.bencode(ping_query)
    
    # Send a datagram to a server and recieve a response.
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    s.sendto(ping_query_bencoded,
             (socket.gethostbyname('router.bittorrent.com'), 6881))
    r = s.recvfrom(1024)
    
    ping_response = bencode.bdecode(r[0])
    
    print(ping_response)

So, I should be able to implement this in PHP, should be fun:D

Bencode

A quick search, and I got this:

composer require rych/bencode

Somebody would call that cheating, I call it: progress 😈

Final ping result

<?php

use Rych\Bencode\Bencode;

include_once __DIR__ . '/vendor/autoload.php';

$socket=@fsockopen("udp://router.bittorrent.com",6881,$err_no,$err_str,1);
if(!$socket) die('socket failed');

$myId = generateId();
print 'ping'.PHP_EOL;
fwrite($socket, Bencode::encode([
    'y' => 'q',
    't' => '0f',
    'q' => 'ping',
    'a' => [
        'id' => $myId
    ]]));
$packetReceived=fread($socket,1024);
var_dump(Bencode::decode($packetReceived));

Jekyll install journey:

I followed these instructions: https://jekyllrb.com/docs/

But, for some reason, I used some old instructions and spent understanding strange ruby/gem errors. Never used it, so it was a mystery to me. But somehow, I got it working.

Then, because I am lazy and sometimes I prefer a nice GUI over editing config files, I Google searched for Jekyll admin. And what I found: https://github.com/jekyll/jekyll-admin

Just added these to Gemfile

gem 'jekyll-admin', group: :jekyll_plugins

And ran these commands:

bundle install
bundle exec jekyll serve --livereload

And after opening: http://localhost:4000/admin/

I was greeted with the similar page:

Well after using it, I will probably try to find something better, because image management for blog posts isn’t smooth. But for now it will be fine.

Time to publish it for free (well I have a domain, but hosting is free)

So how do we get free hosting, that we can trust (well not fully, but it should be stable enough for me)

It’s github, you will ask how? Github is just for source code, you still need hosting. But nah, github has a neat feature called ‘Pages’.

I will create, github public project here: https://github.com/Markomas/blogas Then I will go to Settings -> Pages

And my domain registrar:

Final result: