Donald Feury

Software Development, Linux, and Gaming

Coming soon


So recently I decided to try adding some background music to my videos so it isn't just me blathering the whole time. Naturally, I turn to my weapon of choice in these cases, ffmpeg.

The music I use for this is from a project called StreamBeats, you should check it out.

Select Random Music

For the first part, I wanted to pick a random music file from a directory and use it later. For this, I wrote a simple script:

#!/usr/bin/env sh

dir=$1
find "$dir" | shuf -n 1 > tracklist.txt

head tracklist.txt
  1. I pass in the directory to use
  2. I list out of contents of that directory using find
  3. I pipe that output to shuf, which randomizes the list of files. With the -n 1 flag, it will output only the first line
  4. I write the output of all that to a text file for reference later, as well as using head to list that file to stdout

Add Music

Time for the ffmpeg magic:

#!/usr/bin/env sh

video=$1
bgm_dir=$2
output=$3
bgm="$(random_music "$bgm_dir")"

ffmpeg -i "$video" -filter_complex \
    "amovie=$bgm:loop=0,volume=0.03[bgm];
    [0:a][bgm]amix[audio]" \
    -map 0:v -map "[audio]" -shortest -c:v copy "$output"
  1. I have three arguments here
    • The video to add the music to
    • The directory you want to pick the random music from
    • The path to write the new file to
  2. We get the music file to load in using the random_music script and save it for later
  3. I'll talk about the important parts of this ffmpeg command
    • amovie=$bgm:loop=0,volume=0.03[bgm]; – this loads the randomly chosen music file to make its audio stream available and with the loop argument set to 0, loops it indefinitely. The volume filter is used to adjust the volume of the music to be more β€œbackground music” appropriate
    • [0:a][bgm]amix[audio] – combines the audio from the video and the newly loaded background music into one audio stream
    • -shortest tells ffmpeg to stop writing data to the file when the shortest stream ends, which, in this case is our video stream. The audio stream technically never ends since it loops forever.

Tada, you should have a new version of your video with the randomly chosen music looping for the duration of the video.

#ffmpeg #videoediting

Cache all the things

Odysee YouTube


The other day I was looking to see how difficult it would be to utilize Redis in a go program, not for a specific use case, just in general. Turns out, its pretty freakin easy with the help of a few packages.

The one I decided to use to get a feel for it was this package called redigo.

Setup your project

If you don't have go modules initialized in your project, go ahead and do that.

go mod init gitlab.com/dak425/golang-redis-example

After modules is setup, we need to grab our redis client package.

go get github.com/gomodule/redigo

Now we should be all ready to go (hah get it, go... I'll stop now)

Me stopping

Connecting to a Redis instance

First, lets connect to our Redis instance.

conn, err := redis.Dial("tcp", "localhost:6379")
if err != nil {
  // Handle error
}
defer conn.Close()

We use the Dial function to connect to the instance, passing in two arguments. The first is the protocol to use and the second is the address to the redis instance you want to connect to.

Of course handle errors as you deem necessary.

We also add a deferred called to conn.Close() to ensure our connection gets cleaned up when the program finishes.

Running Redis commands

Setting a value

The redigo package doesn't give us command specific functions like HMSet or something like that. It simply gives us a method on our connection object called Do that allows us to build and send a redis command.

_, err = conn.Do(
  "HMSET",
  "podcast:1",
  "title",
  "Tech Over Tea",
  "creator",
  "Brodie Robertson",
  "category",
  "technology",
  "membership_fee",
  9.99,
)
if err != nil {
  // Handle error
}

I'm building a redis command to set multiple fields on a hash with the key of podcast:1

The Do method returns a reply and an error. I wasn't interested in the reply, only if there was a problem setting the hash fields for that key.

Tech Over Tea is an actual podcast done by a guy I've talked to named Brodie Robertson. He tries to mostly stay on tech centered subjects like Linux and crypto.

I went on it a few months ago, you can find the episode here

He has his own channel where he showcases a bunch of neat little Linux programs and utilities. You can check it out here

Reading key values

Now that we have an entry in our Redis instance, lets read some data back out of it.

First, lets get a single field from our hash we just put in there.

title, err := redis.String(conn.Do("HGET", "podcast:1", "title"))
if err != nil {
  // handle error
}

We're using Do again to build and send a redis command but you'll notice its being wrapped in a function called String. What string does it attempt to convert the redis reply into a go string. If it can't, it will return an error that we can check.

Next, lets grab ALL the entire hash.

podcastHash, err := redis.StringMap(conn.Do("HGETALL", "podcast:1"))
if err != nil {
  // handle error
}

We use another of those type conversion functions called StringMap, which, if successful returns us a map[string]string that we can iterate over.

Now, that's fine but you know what would be really nice? If we could unmarshal our redis response into a struct we had defined. Turns out we can do that.

I have a podcast defined here for use.

type Podcast struct {
  Title    string  `redis:"title"`
  Creator  string  `redis:"creator"`
  Category string  `redis:"category"`
  Fee      float64 `redis:"membership_fee"`
}

You'll notice we've given the struct properties a tag. This tag, not unlike the json tag, is used by certain packages to know how to unmarshal our redis reply into our struct's properties.

So how do we unmarshal our reply into the struct using this package? Well it is a two step process. First, we use the Values function from the redigo package to convert our reply into a []interface{}.

values, err := redis.Values(conn.Do("HGETALL", "podcast:1"))
if err != nil {
  // handle error
}

After we have our converted response, we create an instance of our struct, and pass both the response and a pointer to our struct instance to a function called ScanStruct.

var podcast Podcast
err = redis.ScanStruct(reply, &podcast)
if err != nil {
  // handle error
}

Assuming no error occurred, if you print your struct using Printf or other fmt functions, you will see that the struct now has the values from our hash!

Summary

Now you should be able to write and read data from your Redis instance with your go programs. There are some more advanced concepts you can look into regarding this package, such as connection pooling.

Thank you for your time and be sure to check out my other go articles on here and my other videos on my YouTube.

#go #golang #redis

Total Control

Odysee YouTube

Check out the video for some more elaboration on the topics below.


If you liked it and want to know when I post more videos, be sure to subscribe

Time to start looking at some ORMs for go, today we're gonna start with gorm. Gorm is a code first ORM, meaning we can use go code to define our database scheme, run the migrations, and also interact with the database with the same code.

Connect to the Database

In order to connect to a database using gorm, we simply use the following:

db, err := gorm.Open("sqlite3", "test.db")
if err != nil {
  // Handle error
}
defer db.Close()

We are opening a sqlite3 file, because I didn't want to fiddle with something like MySQL for this.

We handle an error if one occurred, and defer a call to db.Close() to ensure the connection is cleaned up afterwards

Defining Schema with Structs

We have the following structs:

type Channel struct {
	gorm.Model
	Name        string
	Description string
}

type User struct {
	gorm.Model
	Email    string
	Username string
}

type Message struct {
	gorm.Model
	Content   string
	UserID    uint
	ChannelID uint
	User      User
	Channel   Channel
}

Each of these structs will have a corresponding table in our database, with the same columns as the structs have properties.

The gorm.Model struct has some standard properties commonly used in SQL database, check out the docs to see more about it.

You'll notice the Message struct has properties that reference other structs. These are used to map the table relations, read more about how gorm does relationship mapping here.

Migrations

The easiest way to sync up our schema to our structs it to use the AutoMigration method defined on the db instance:

db.AutoMigrate(&Channel{}, &User{}, &Message{})

We pass in an instance of each struct and gorm does some reflection under the hood to make any schema changes, generate the queries, and run them.

Keep in mind, AutoMigrate will only add missing tables, columns, and indexes. It will not remove unused ones, read why here.

Adding Data

To create a new row in our database, we simply create in instance of one of our structs, set the values on its properties, and pass it to the Create method on our database connection:

channel := Channel{Name: "General", Description: "General banter"}

db.Create(&channel)

Tada, data has been inserted into our table!

Finding Data

If you want to grab something from the database, the docs has alot of examples but some notable ones are as follows:

// give me all the channels and unmarshal them into my variable channels, which is of type []Channel
db.Find(&channels)

// give me the first record by primary key, in the channels table and unmarshal it into my variable channel, which is of type Channel
db.First(&channel)

// same as db.First() but gets the last record by primary key
db.Last(&channel)

You can also build where clauses using the Where method:

// Adds a 'where name = "General"' clause to the select query
db.Where("Name = ?", "General").Find(&channels)

// You can pass in structs to act as the where clause source.
// Here it will take the field 'Name' from our channel struct and add it into the where clause
db.Where(&Channel{Name: "General"}).First(&channel)

Error Handling

Gorm handles errors in a way that is a little different than idiomatic go.

After you run a gorm query, it will set a value to the Error variable in the gorm package, if an error occurred while running the query.

Here is an example of checking if the method ended up returning an error:

if err := db.Where("Name = ?", "MissingName").First(&channel).Error; err != nil {
  // Handle error
}

Summary

Gorm makes it fairly easy to manage your database schema and acts as an abstraction over your database backend.

Thank you for reading

#go #golang #orm #gorm #databases

Maps those objects

Odysee YouTube

Check out the video for some more elaboration on the topics below.


If you liked it and want to know when I post more videos, be sure to subscribe

I'm about to do a few videos on a some Go ORM packages and thought it wouldn't hurt to do a dedicated segment on just talking about what ORMs are and why you should or shouldn't use them.

What does ORM stand for?

ORM stands for Object Relation Mapping. Typically this means communicating with a system using a language other than the native language is expects.

An example of this would be a SQL database. A SQL database is expecting, well, a SQL query to interact with it, but what if we wanted to interact with it with something like a Golang program?

What does an ORM do?

An ORM library gives us the mechanism by which to perform Object Relation Mapping. This means we end up with structs or classes that represent something like a table in our database

In golang, we would get something like this:

user := models.Users().ByID(1)

Which would generate the following SQL query:

SELECT * FROM Users WHERE id = 1;

Pros & Cons of ORMs

Pros:

  • Much less time spent to interact with a database in your program
  • Abstracts away the database being used, which makes it easier to swap to another backend
  • If you have weak SQL skills, the generated queries are at least as good as if you wrote them, if not more performant.

Cons:

  • If you need a very highly optimized query and you can write said query, it may perform better than the generated ones.
  • There is some amount of mental overhead related to learning an ORM library
  • Most ORMs require some amount of configuration
  • May not help you developer stronger database and/or SQL skills

What kinds of ORM libraries exist?

From my experience, there are two primary types of ORM libaries

Code-First ORM

A code first ORM uses the code written or generated by the user to generate the database schema and applies the schema to the database.

Some examples of code first ORMs:

  • Gorm (Go)
  • Basically every ORM in most mainstream framework
    • Eloquent (Laravel)
    • ActiveRecord (RoR)
    • Whatever Django uses

Schema-First ORM

A schema first ORM reads the already defined schema from the database and generates from it, all the code necessary to interact with the database.

Some examples of schema first ORMs:

When to choose which?

Code-First ORM

  • Most ORM libraries in my experience are code first, so a lot of choices
  • These tend to do alot, some code generation, acts as abstraction, and manages migrations (schema changes)

Schema First ORM

  • Need to get up and running quickly with an existing database (legacy data)
  • Almost ALL of the code will be generated, vs just some being boilerplated like most code first ORMs I've used
  • You prefer a more unix approach, as most schema first ORMs I've seen don't handle migrations. You'll need to use a seperate tool or library to manage migrations.

#databases #orm

Take Command With Cobra

Odysee YouTube


Shout out to my wife who makes these hilarious thumbnails, follow her on twitter for more art stuff.

CLI applications are extremely handy to use. Building out alot of the nice features that good CLI apps have can be a bit time consuming though.

At least, it was, until cobra was created and that changed everything.

Cobra is a go package that can be utilized to create very robust CLI applications with very little effort.

It provides an impressive list of features: – Subcommand-based CLIs – POSIX-compliant flags (short/long versions) – Nested subcommands – Global, local, and cascading flags (explained in video) – Application and command generator – Auto suggestions on command typos – Automatic help generation – Automatic help flag recognition – Generates shell autocompletion for bash, zsh, fish, and powershell – Generates documentation, such as man pages – Command aliases – Integrates very well with viper

Check out the video tutorial where I give some basic examples of the various features of cobra.

If you liked it and want to know when I post more, be sure to subscribe and thank ya'll again for your time!

#go #golang #cobra

Configure Go with Viper

YouTube Video

Shout out to my wife who makes these hilarious thumbnails, follow her on twitter for more art stuff.

You can get quite far configuring your go programs with the standard library. You can read in your config file, or pass in arguments and parse them with the flag package. However, what you wanted to do all these things interchangeably and with much more ease?

Enter viper, go configuration with fangs as they say! With viper, you can do ALL of the following:

  • setting default values for your configuration
  • read from popular config file formats, such as JSON, TOML, and YAML
  • watch your config file and live reload any changes
  • reading from environment variables
  • reading from remote config systems, such as etcd
  • reading from command line flags
  • reading from buffer
  • setting explicit values

Check out the video to see an example of some of things you can do with viper.

If you liked it and want to know when I post more, be sure to subscribe and thank ya'll again for your time!

#go #golang #viper

Error Chains

YouTube Video

Go added a neat feature in 1.13 that they call error chains

Errors can now implement a method called Unwrap that should return another error. Typically, this is an error that occurred further down in the call stack.

It is very common in go to have errors get tossed further and further back up the call stack as functions resolve, until its finally handled properly. Error chains allow you to carry a series of errors all the way back up the call stack more easily.

They added two new functions in the errors package to interact with this concept, Is and As

Is takes a argument of an error and another error, returns true if any instance of the second error occurs in the first error's chain, and false otherwise.

As takes the same arguments, but sets the value of the second argument equal to the error that was found in the first error's chain if it was present.

Check out the video to see the concept in action.

If you liked it and want to know when I post more, be sure to subscribe and thank ya'll again for your time!

#go #golang #errors

Go Error Handling

YouTube Video

Error handling in Go is pretty straight forward. There are no exceptions, there are no try catch blocks, there is only ERROR.

Error is a type like any other primitive built into the language, like a int or a string. This makes error handling, while a tad verbose, very straight forward and easy to understand.

Check it out and let me know what ya'll think. Any feedback is greatly appreciated.

If you liked it and want to know when I post more, be sure to subscribe and thank ya'll again for your time!

#go #golang #errors

Go Templates

YouTube Video

Code can be found here

Go actually has a template package in its standard library, which is pretty sweet. Normally, this is something you see in separate 3rd party libraries or included in full frameworks, such as Laravel or Rails.

Its very powerful too. In addition to injecting your data into templates, you can extend what you can do while processing the data passed into a template by passing in a map of functions.

I once used it to generate the approx. 150 data types that were listed in Mixer's API documentation. That saved me tons of time from writing boilerplate code.

Check it out and let me know what ya'll think. Any feedback is greatly appreciated.

If you liked it and want to know when I post more, be sure to subscribe and thank ya'll again for your time!

#go #golang #templates

Using MySQL with Go

Youtube Video

Code can be found here

Time to start integrating some other tools with our go programs.

I figured a good place to start would be with a SQL database, since its very common to work with one. I chose MySQL for this example, as I have a lot of background with it from my PHP days.

We don't do anything crazy here, just demonstrate how to load the driver, connect to the database, and perform simple queries and prepared statements.

Check it out and let me know what ya'll think. Any feedback is greatly appreciated.

If you liked it and want to know when I post more, be sure to subscribe and thank ya'll again for your time!

#databases #go #golang #mysql