It is an essential part of the standard experience expected of a CLI app.

One such example is the cURL command. Following is a gif of the difference in output when using the curl command with the verbose flag and without.

Today we'll try to see how to implement such a feature in a CLI app using Go.

Defining the feature

Let's define the flag we are going to implement

• It should work for all commands and subcommands in the application

• When provided, it should show additional logs of all the commands, in addition, to the output that is shown by default.

• It should be easy to maintain, every command or every log statement must not need to handle or know about this flag

Luckily for us, Go has amazing standard packages we can compose to create this effect with ease.

Tres Amigos

We will use three packages to accomplish this.

1. flag: We will use it to easily parse the arguments provided by the user

2. log: We will use the standard package to log stuff

3. io: We will utilise a very special Writer provided by this package.

flag

The flag parsing works in three steps which need to be followed in the exact same order

1. Define the expected flags

2. Invoke the Parse() method

3. Start accessing the values

func main() {
    // Define the flag that is expected 
    // We can pass the name, a default value & a description
    // It will return a pointer which is guranteed to have value 
    // Only after flag.Parse is called
    verboseP := flag.Bool("v", false, "Enable verbosity")

    // From the docs: Must be called after all flags are defined 
    // and before flags are accessed by the program.
    flag.Parse()

    // Access the value after 
    isVerbose := *verboseP
}

log

Log package contains a function called SetOutput which can be set to anything that implements the Writer interface.

We can use this to redirect all the logs to a specific writer.

log.setOutput(SomeWriter)

io

IO will provide us with the last missing piece, Writer which does nothing.

Basically discards any logs sent to it. It's called io.Discard

Putting it all together

package main

import (
    "flag"
    "fmt"
    "io"
    "log"
)

func main() {
    // Define the flags
    verboseP := flag.Bool("v", false, "Enable verbosity")

    // Invoke the parser
    flag.Parse()

    // Access it 
    isVerbose := *verboseP

    // If verbosity is not required discard all upcoming logs
    if !isVerbose {
        log.SetOutput(io.Discard)
    }

    fmt.Println("Hello world!")
    log.Println("I'm tired of printing this")
}

Try it on repl.it

I’m not talking about the scenario where you expect a certain result and get something else; that might be related more to the code you personally wrote.

I’m specifically talking of a scenario when you either got started up at a new place with a completely new project; maybe the framework is new for you; maybe they have a custom-made framework for their org.

One day while working on a task; you somehow introduced a bug that throws some sort of exception which you have no idea why it came nor can you deduce how your code could have broken anything. You don’t even know the codebase well enough to be able to figure out remotely what could’ve caused it.

During a time like this, you can either get overwhelmed by that 200 lines of errors; or calmly befriend them. As it is the only thing that can help you understand the issue the quickest; not solve, but understand.

Error messages might not specifically help you fix the issue or come up with a solution, but they’re a great tool to understand what went wrong.

In layman terms, the gibberish that gets printed when your code fails is known as a stack trace.

Understanding Stack-Traces

For example, you have a piece of code that starts executing from a file called A, calls a function in file B, and B, in turn, calls a function in C; something goes wrong in C and it throws an exception Ec; which if not handled, will result in the program to exit with non-zero code. For such a scenario, you can always assume that the info about C will come first followed by B and A.

There’s a very special reason for this as well; since thinking outside-in in a scenario like this wouldn’t really help much for a developer to understand the context as well. As when you start from the outside you have a lot of context to understand and process till you reach that line that actually throws an error. And also have to think about code that might not even be related to the error you’re trying to debug.

So the better way is to look inside-out; you start with looking at the most minimal context and try to figure out the reason why it failed; failed to understand? Ok, increase a level of context, now you can look at B which is called C, and figure out what might’ve gone wrong there, and just move contexts up and gradually build up on it until you figure out what actually went wrong.

What does it look like?

Generally speaking, a stack trace will start with the name of an exception(or error); it will provide some background information about it and then there will be N number of lines starting from where the exception originated in leading up to where the execution began from.

Most probably, each line of the stack trace will also contain some information about which function/method the exception originated from, and additionally also provide information about the file and even the line number where it happened.

For example:

caused by SomeException
    at someClass.someFunction(SomeFileName.someExt:lineNumber)
    at someClass.someFunction(SomeFileName.someExt:lineNumber)

Needle in a haystack - Recognising lines of interest

I feel like stating the obvious, but the quickest way to understand the reason why some error came lies in trying to understand the flow of control leading up to the error; i.e. how did the flow of control reach that point of failure; and what was the path in terms of lines of code it travels.

However, it may not seem obvious when we think about how modern software development works; we mostly use a framework or a bunch of library functions to get started. And while we try to manifest the app we want to create, we often end up using a lot of functions that we ourselves have not written; aka library code.

So, realistically in a stack trace, you can expect only at most 5 lines in a stack trace to be significant enough for you to pay attention to; but those 5 lines will be hidden within a plethora of methods from the framework or library which is being used.

So, the lines you need to be paying attention to are basically lines that are from your project; as those are the only lines where you might have added something which would break the app.

This is where IDEs can be of good help; most good IDE has this amazing feature where while showing errors they highlight all the files that belong to your project and are written by you as well as gives a link over it, upon clicking which the exact file and line is opened on that IDE.

In the above image, we can see a stack trace of what seems like a validatePassword gone wrong. You can very easily just glance and locate the orange lines, and you will notice that all of those lines start with com.company which is the package name for the current project.

This is how IDEs may help in this first step of zoning into specific lines from a bigger stack trace.

Avoid this silly mistake

A lot of people; to be ‘safe’ tend to wrap their code in try-catch blocks while catching the java.lang.Exception class. This is scary; as now all the exceptions the programmer hasn’t even thought about getting ‘caught’ and since the bubbling of that event is stopped, it gets very hard to catch or debug that issue. As there will be no stacktrace for it at all.

I’m talking about a code somewhat like this

public void doSomething() {
    try {
        someService.createSomeThing();
    catch (Exception ex) {
        logger.error("Something went wrong", ex);
    }
}

If the code is like this; and somehow the database goes down, instead of throwing a scary 500 which will prompt everyone to quickly look at it, it will silently handle that error and just provide a log statement as a response. Even while trying to investigate in such a scenario when someone will look into the logs they will only see the classname of the error that happened, and that’s it.

One thing that can be done(not should) is on top of just printing the ex we can also log ex.getStackTrace() to be sure that in such scenarios atleast the whole stacktrace is logged for us to then read and understand the issue.

However, what I think should be done is we should only handle exceptions that we expect; anything else that we didn’t probably means it is something that should fail our app.

As an example; let’s say you have a createUserName method which creates a userName for a specific user. If you know it can only throw two exceptions, UserNameAlreadyExists or UserNameFormatInvalid; then you should only handle those two exceptions and handle them accordingly and not catch java.lang.Exception as that means you will literally catch all exceptions that might happen downstream.

Find out exactly what to google

Let’s say now you’ve gotten comfortable at looking at stacktrace, and you know where to look at and also how to locate lines relevant for you, the next simple step is googling. Learning how to read stacktrace will give you exactly what you need to google to understand the issue.

For example; let’s say you have a big stacktrace, and you locate 3lines of it where your own code is mentioned, and at the topmost entry an exception is thrown that you have never heard of, this is the time to just google the specific name, learn about it and think how your code could have thrown it; as massive hints you now know exactly which lines to look at in exactly which files; which will make this process a bit faster.

Closing thoughts

This was a very basic guide; however every language and framework has a few things slightly different and you might need to fine-tune those while working to be at an advantage; for example with Laravel; whenever a 500 is thrown in a debug-enabled environment, the error page includes all the environment variables that were set and all the request params that came in for the request. So if you know about that, debugging an issue that is related to a wrong or cached environment variable will be a breeze.

Discovering co-pilot

I started using copilot maybe within 3months of them starting to roll it out; during the time I was working on a fresh new backend project. Which was built using Typescript. For a while, after installing it I was reluctant to use it and had only tried it out a few times for just creating a function to add two numbers and stuff.

I did try to get into it though, but found it rather slow and eventually disabled it and moved on.

Until one fine day, I had a bit of extra work and needed to squeeze it in somehow when I remembered this tool and it immediately was so much better than the last time I had used it; it was predicting correctly and quickly.

Since then I had used co-pilot extensively for a lot of my work and hobby projects.

My experience using it

I have actually used co-pilot to write large chunks of backend code; basically reading from tables, fetching from tables, query generation, converting queries into knex implementations; and the list goes on.

And I’ve messed up a lot of times as well, you see, just like autocomplete, co-pilot also impairs you someway, very slowly you start giving in to it, and start letting go of the cognitive load cause now you only think one level of abstraction above the actual implementation, you start thinking in terms of specification rather than implementation.

Let me explain, during this time, I noticed myself getting into bugs that I would normally never get into, and stuff that was really hard to explain at first, almost one of those times you’d perform the sin of questioning the compiler. Almost all of those bugs were small typos which I’d miss because I overlooked them as I myself didn't write any of those 20LOC.

Things co-pilot is really good at

• Converting very specific instruction in english into code, the less the abstraction the better
• This means, it's really good at writing singular units of code where context is limited to it’s scope
• It is also good at picking up additional information present in the file, i.e. it will intelligently pick up which variables must be used in the next operation and which other functions might be called as a side effect of the current one
• Commenting and documenting your code, you can expect to start documenting a piece of code and you’ll find co-pilot brilliantly fill in the details for you by understanding the implementation
• You can sort of get it to do chores for you as well, I had a bunch of code where there was a SQL prepared statement and I had the task to refactor it out, and co-pilot helped beautifully. Ever since then, I got into a habit of playing with queries in the MySQL console, pasting it in the ide, and making co-pilot write the knex equivalent for it.

Things it isn’t good at

• In my personal opinion, co-pilot isn’t very good with magic code, i.e. code that is heavily abstracted
• It isn’t good with libraries or patterns which are uncommon. Let’s say your org has a big internal library which is very unique, you can’t really expect copilot to predict it that well; however, if you have similar implementations in the same file you can start typing it out and co-pilot sorta fizzy finds it’s way to whichever implementation you'd wanna use.

Best use cases right now

• Use it as google/StackOverflow; whenever there’s a piece of code that you have already written once in your life, and know how it works you can easily describe it as a comment and co-pilot should lead you there. The co-pilot is a beast at this.
• Refactor and chores. First of all, make sure you read each and every piece of code generated by this, as small diversions from the expected code can make the code behave in ways you wouldn’t be able to catch very quickly, and it should be tough as you are not the person who implemented it. I’d say co-pilot mostly gets it right, even contextual information, I’ve seen it refactor a nested SQL query into beautiful knex.

Examples

Example 1

In the gif below, you'll see me asking co-pilot to create a function which gets users from somewhere, the most common usecase being fetching it from a relational database. After the first suggestion, you'll notice I changed the function signature a bit hinting at a possible pagination implementation, after which co-pilot very quickly understood the context and implemented the pagination code in the query on it's own.

This is how you can calibrate in scenarios where you need an unconventional or uncommon code you can either open up the co-pilot suggestion list (Ctrl Enter) and pick one that's matches.

If you don't find the exact code you're looking for, get one that is closest and then just write the rest on your own, however, if you don't even find a resembling piece then maybe you're describing a really big and abstract code and maybe you can try breaking it into subtasks and asking github to fill in the code per atomic operation.

Example 2

In the gif below, you will see I am not making co-pilot write a new piece of code, rather making it do refactoring work, I really enjoy taking it's help while doing stuff which is necessary but sometimes boring.

Example 3

In the gif below, you'll see that I wanted to have a function which uploads a local file to a s3 bucket, however I wanted to implement it using boto3.resource and not boto3.client; in this scenario changing the function definition doesn't make sense, so we add a comment to guide co-pilot into the right direction.

Closing thoughts

After using co-pilot for a while, you guys will learn this sense of guiding copilot; I’ve eventually developed this way of setting up the stage around the part of my file where I’d want co-pilot to fill in some lines, and if you do the setup just right, you can guide co-pilot really well to predict the exact code that you’d want to generate.