Programming

More on validating with Go

A few days ago I posted about a new Go package called validator that we initially developed for our own internal use at project7.io but what fun is internal stuff, right? So we opensourced it.

Then Matthew Holt pointed me to an ongoing discussion
on validation happening on martini-contrib’s github. By the way, if you don’t know martini yet, go rectify that right now.

And today I learned about check, which takes a completely different
approach to data validation than the one we took, which by the way is totally fine and you should check it out.

Good to know we were not the only ones with the problem. Although I’m happy with how our package turned out, I kind of wish
I had found these other guys earlier. Could have shared some code or just ideas.

[ANN] Package validator

The thing about working in a startup under stealth mode is you can’t often talk about what you’re doing. Thankfully, from time to time, an opportunity appears that lets us at least share something tangential.

A large part of our project at project7.io involves receiving data from a client (generally in JSON) for processing. This involves unmarshaling the JSON into a struct, something that Go does very well. But then comes the boring part: checking that the information sent from the client is correct and complete before doing anything with it.

The boring life of validating stuff

1
<span class='line'><span class="k">if</span> <span class="nx">t</span><span class="p">.</span><span class="nx">Username</span> <span class="o">!=</span> <span class="s">""</span> <span class="o">&&</span> <span class="nx">t</span><span class="p">.</span><span class="nx">Age</span> <span class="p">></span> <span class="mi">18</span> <span class="o">&&</span> <span class="nx">t</span><span class="p">.</span><span class="nx">Foo</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="o">&&</span> <span class="nb">len</span><span class="p">(</span><span class="nx">t</span><span class="p">.</span><span class="nx">Bar</span><span class="p">.</span><span class="nx">Baz</span><span class="p">)</span> <span class="p">></span> <span class="mi">8</span> <span class="o">&&</span> <span class="o">...</span>
</span>

We had to do this often and for a large number of different structs and sometimes a struct gained a new field and we had to go back and see that it was being properly validated everywhere. It was so boring that we ended up writing something to make it easier for us. We’ve been using this for a while and now we decided to open source it in the hopes that it might be useful for others.

Package validator implements validation of variables. Initially we had implemented JSONschema but we don’t always deal with JSON, we also get data as XML and sometimes as form-encoded. So we changed our approach and went right to the struct definitions.

A struct definition with some validation rules attached

1
2
3
4
5
6
7
8
<span class='line'><span class="kd">type</span> <span class="nx">T</span> <span class="kd">struct</span> <span class="p">{</span>
</span><span class='line'>  <span class="nx">A</span> <span class="kt">int</span>    <span class="s">`validate:"nonzero"`</span>
</span><span class='line'>  <span class="nx">B</span> <span class="kt">string</span> <span class="s">`validate:"nonzero,max=10"`</span>
</span><span class='line'>  <span class="nx">C</span> <span class="kd">struct</span> <span class="p">{</span>
</span><span class='line'>      <span class="nx">Ca</span> <span class="kt">int</span>    <span class="s">`validate:"min=5,max=10"`</span>
</span><span class='line'>      <span class="nx">Cb</span> <span class="kt">string</span> <span class="s">`validate:"min=8, regexp:^[a-zA-Z]+"`</span>
</span><span class='line'>  <span class="p">}</span>
</span><span class='line'><span class="p">}</span>
</span>

This allowed us to attach validation rules right to the data structure definitions. Then instead of large, boring list of if statements, we were able to validate in single function call.

Validating an instance of a struct

1
2
3
<span class='line'><span class="k">if</span> <span class="nx">valid</span><span class="p">,</span> <span class="nx">_</span> <span class="o">:=</span> <span class="nx">validator</span><span class="p">.</span><span class="nx">Validate</span><span class="p">(</span><span class="nx">t</span><span class="p">);</span> <span class="p">!</span><span class="nx">valid</span> <span class="p">{</span>
</span><span class='line'>  <span class="c1">// not valid, so return an http.StatusBadRequest of something</span>
</span><span class='line'><span class="p">}</span>
</span>

Multiple rules for different situations

We often use the same struct to deal with different data coming from the client. Sometimes we only care about one or two of the fields in one scenario, so we also supported multiple rules like, say

A struct with two different sets of rules

1
2
3
4
<span class='line'><span class="kd">type</span> <span class="nx">T</span> <span class="kd">struct</span> <span class="p">{</span>
</span><span class='line'>  <span class="nx">A</span> <span class="kt">int</span>    <span class="s">`foo:"nonzero" bar:"min=5,max=10"`</span>
</span><span class='line'>  <span class="nx">B</span> <span class="kt">string</span> <span class="s">`bar:"nonzero"`</span>
</span><span class='line'><span class="p">}</span>
</span>

In scenario foo, we need A to be non-zero, but we don’t care for what is in B. In the bar scenario, however, we need B to be non-zero and the value of A to be between 5 and 10, inclusive. To validate, we then make validator use a different tag name for each case

WithTag() FTW

1
2
3
<span class='line'><span class="nx">t</span> <span class="o">:=</span> <span class="nx">T</span><span class="p">{</span><span class="nx">A</span><span class="p">:</span> <span class="mi">3</span><span class="p">}</span>
</span><span class='line'><span class="nx">validator</span><span class="p">.</span><span class="nx">WithTag</span><span class="p">(</span><span class="s">"foo"</span><span class="p">).</span><span class="nx">Validate</span><span class="p">(</span><span class="nx">t</span><span class="p">)</span> <span class="c1">// valid</span>
</span><span class='line'><span class="nx">validator</span><span class="p">.</span><span class="nx">WithTag</span><span class="p">(</span><span class="s">"bar"</span><span class="p">).</span><span class="nx">Validate</span><span class="p">(</span><span class="nx">t</span><span class="p">)</span> <span class="c1">// invalid</span>
</span>

We can also change the tag by using SetTag with will then make the tag name persistent until changed again by another call to SetTag.

Please refer to http://godoc.org/gopkg.in/validator.v1 for a lot more documentation, use cases, and how to access the individual validation errors found for each field.

Linux Kernel Linked List Explained

I appreciate beautiful, readable code. And if someone were to ask me for an example of beautiful code, I’ve always had the answer ready: the linked list implementation in the Linux kernel.

The code is gorgeous in its simplicity, clarity, and amazing flexibility. If there’s ever a museum for code, this belongs there. It is a masterpiece of the craft.

I was just telling a friend about it while we talked about beautiful code and he found this piece that I share here: Linux Kernel Linked List Explained.

Container changes in C++11

The recently approved C++11 standard brings a lot of welcome changes to C++ that modernize the language a little bit. Among the many changes, we find that containers have received some special love.

Initialization

C++ was long behind modern languages when it came to initializing containers. While you could do

[cpp]int a[] = {1, 2, 3};[/cpp]

for simple arrays, things tended to get more verbose for more complex containers:

[cpp]
vector v;
v.push_back(“One”);
v.push_back(“Two”);
v.push_back(“Three”);
[/cpp]

C++11 has introduced an easier, simpler way to initialize this:

[cpp]
vector v = {“One”, “Two”, “Three”};
[/cpp]

The effects of the changes are even better for things like maps, which could get cumbersome quickly:

[cpp]
map<string, vector > m;
vector v1;
v1.push_back(“A”);
v1.push_back(“B”);
v1.push_back(“C”);

vector v2;
v2.push_back(“A”);
v2.push_back(“B”);
v2.push_back(“C”);

m[“One”] = v1;
m[“Two”] = v2;
[/cpp]

This can now be expressed as:

[cpp]
map<string, vector> m = One,

                             {"Two", {"Z", "Y", "X"}}};

[/cpp]

Much simpler and in line with most modern languages. As an aside, there’s another change in C++11 that would be easy to miss in the code above. The declaration

[cpp]map<string, vector> m;[/cpp]

was illegal until now due to » always being evaluated to the right-shift operator; a space would always be required, like

[cpp]map<string, vector > m[/cpp]

No longer the case.

Iterating

Iterating through containers was also inconvenient. Iterating the simple vector v above:

[cpp]
for (vector::iterator i = v.begin();

 i != v.end(); i++)
cout << i << endl;[/cpp]

Modern languages have long had some foreach equivalent that allowed us easier ways to iterate through these structures without having to explicitly worry about iterators types. C++11 is finally catching up:

[cpp]
for (string s : v)

cout << s << endl;

[/cpp]

As well, C++11 brings in a new keyword, auto, that will evaluate to a type in compile-type. So instead of

[cpp]
for (map<string, vector >::iterator i = m.begin();

 i != m.end(); i++) {

[/cpp]

we can now write

[cpp]
for (auto i = m.begin(); i != m.end(); i++) {
[/cpp]

and auto will evaluate to map<string, vector>::iterator.

Combining these changes, we move from the horrendous

[cpp]
for (map<string, vector >::iterator i = m.begin();

 i != m.end(); i++)
for (vector<string>::iterator j = i->second.begin();
     j != i->second.end(); j++)
    cout << i->first << ': ' << *j << endl;

[/cpp]

to the much simpler

[cpp]
for (auto i : m)

for (auto j : i.second)
    cout << i.first << ': ' << j << endl;

[/cpp]

Not bad.

C++11 support varies a lot from compiler to compiler, but all of the changes above are already supported in the latest versions of GCC, LLVM, and MSVC compilers.

FOR is evil or something

Have you ever wondered how FORs impact your code? How they are limiting your design and more important how they are transforming your code into an amount of lines without any human meaning?

How can you not want to read an article that starts like that? I had to steal the intro from the original. Seriously, I have used FOR since I learned BASIC back in the day. I never thought about how it was limiting my design. I. Must. Learn. How.

The article I am referring to is, Avoiding FORs – Anti-If Campaign. Eager learner I, I could not not read it.

After the resplendent intro, Avoiding FOR goes on to show “how to transform a simple example of a for […], to something more readable and well designed.”

It takes this unreadable piece of code — that I now recognize as unreadable:

[java]
public class Department {

private List resources = new ArrayList();

public void addResource(Resource resource) {
    this.resources.add(resource);
}

public void printSlips() {

    for (Resource resource : resources) {
        if(resource.lastContract().deadline().after(new Date())) {

            System.out.println(resource.name());
            System.out.println(resource.salary());
        }
    }
}

}[/java]

My eyes hurt already. Thankfully the author transforms the aberration above into this clean, much more readable snippet:

[java]public class ResourceOrderedCollection {

    private Collection<Resource> resources = new ArrayList<Resource>();



public ResourceOrderedCollection() {
    super();
}

public ResourceOrderedCollection(Collection<Resource> resources) {
    this.resources = resources;
}

public void add(Resource resource) {
    this.resources.add(resource);
}

public void forEachDo(Block block) {
    Iterator<Resource> iterator = resources.iterator();

    while(iterator.hasNext()) {
        block.evaluate(iterator.next());
    }

}

public ResourceOrderedCollection select(Predicate predicate) {

    ResourceOrderedCollection resourceOrderedCollection = new ResourceOrderedCollection();

    Iterator<Resource> iterator = resources.iterator();

    while(iterator.hasNext()) {
        Resource resource = iterator.next();
        if(predicate.is(resource)) {
            resourceOrderedCollection.add(resource);
        }
    }

    return resourceOrderedCollection;
}

}

public class Department {

private List<Resource> resources = new ArrayList<Resource>();

public void addResource(Resource resource) {
    this.resources.add(resource);
}

public void printSlips() {
    new ResourceOrderedCollection(this.resources).select(new InForcePredicate()).forEachDo(new PrintSlip());
}

}[/java]

Wait, what? Is this an Onion article?

Snarky Mode Off.

I understand what the author wanted to do, but really, the example used is so off the left field that it’s not even funny.