5/20/2020 | Start with SELECT *

Start with Data. If there is no description of the system you are trying to grok - look at the data that comes in and out. If you don't have data in the system - think about what data needs to go in and out.

A simple "SELECT * FROM X" can save you and your coworkers lots of time and energy.

I'm getting started on an ETL project that a couple of architects have been designing the past few weeks. We are lacking documentation. The schema in the database is still being fleshed out. However, we have a bit of test data in the DB.

For the piece I am responsible for, I want to understand what kind of database models I'll be working with. Without a description of the schema and what each column represents, I need help to understand. I almost reached out to one of the staff-level engineers to get some insight, when I realized I should probably just look at the underlying mock data to understand the relationships.

This reminded me of my journey from a junior to senior engineer. Juniors need to be told how something works. Seniors can use code, docs, and data for understanding. In the old days, I didn't look at the Data. I was overwhelmed with the number of tables. Their names. And their column names. I never bothered to look inside those tables with a "SELECT *". I could have saved a huge amount of time by sitting down and going through the entire database with "SELECT *".

Although this ETL project isn't using Clojure, I'm constantly being reminded of the superpowers embedded in lisps. We are doing ETL and supporting a DSL written on top of a statically-typed, compiled language. If you want to understand the power of Clojure (any lisp really), work in those two domains using a typed, compiled language (lots of reflection, and lots of waiting for the compiler). I'm also reminded of Rich Hickey's talk on the essence of data.

5/3/2020 | Enterprise Dev to Self Employed, 5 month followup Wow! My original post (below), got over 1,000 views, opened many new connections, and I got to respond to everybody that wrote me on email (which I encourage you to do as well!). It's been about 4.5 months, since I wrote that, and I've experienced a couple of months of React contracting and a lot of C# & .NET Blazor work! It's been a success, even though I had to sell my car to make the first couple of months happen. My results were 2 jobs landed, 1 outbound and 1 inbound. The data: I posted once on the HN freelancer thread in October, and once in November. I reached out to about 5 people I could have helped out, 1 turned into a gig. 1 was a really sketchy person who insisted everything was under NDA and he could only pay me in crypto, so I avoided going further down that path. Lastly, I got 1 inbound gig, that turned into a full time job (which I wasn't aiming for, but couldn't pass up). Breaking it down further, I got the first gig by reaching out to someone who was seeking an intermediate android developer. We scheduled a call, chatted, happened to have a lot in common and he ended up letting me work on another project instead which wasn't advertised, and ended up being React! The second gig was really more of a take home job interview project that I got paid for. I had to build a pretty basic CRUD app to show that I knew what I was doing. Importantly, the only reason I even got the "interview" was because I had a Twitter. On Twitter, many months before, I had decided I wanted to get really good with .NET Blazor, a new Single Page App paradigm, so I put ".NET Blazor expert" in my Twitter bio as an aspirational public goal. I had no itention of anyone hiring me based on that data point, but it just so happened that my boss was looking for a Blazor person. I was totally forthcoming about my actual level of Blazor skill once we talked and he said he needed me for Blazor. The biggest takeaway from following and generating leads so far, has been simple: It doesn't happen like you anticipate. Both gigs were totally indirect opportunities, but I can say that it helped that I had a twitter and blog tied directly to my name and hacker news profile. Now, the actual job - I've been a .NET Blazor "expert" since early January this year, and to be honest, at the pace my team works at, I could easily be in the top percentage of Blazor server developers just because we work rapidly and only make new things! No rewrites means I have cranked out many new features each week since January, which counts for something. However, I still have a long way to go in the .NET ecosystem, and my coworkers literally have 10x the amount of experience in .NET than I do, so they are truly experts on the platform. Anyways, I was more than happy to take this as a full time job, purely for the fact that my coworkers are experts. It's just icing on the cake that we use cutting edge technology, only work on MVP's, and are about to build a product in our spare time. Oh, and the pay is better than my big-corp job and it's remote. These are the reasons I will intentionally be ignorant of my selection bias and luck, and recommend for others to follow suit and leave big-corp America! There are sweet opportunities out there if you are willing to give up some safety. Some of you might have better luck, and others might take a bit longer to find an opportunity. Of course, you can always try to get consulting going in the evenings while you have your job, so you can minimize the risk. The sad part of this story is that I have not been able to work with Clojure for $ yet. I'm still holding onto hope, and maybe I can even convince my boss to use it on a product, but until then, I'll just fire up emacs and toy with it here and there. The last thing I want to say is that I haven't invested much energy into my twitter, which I'm trying to grow. I did take down my aspirational title of ".NET Blazor Expert", but as part of my online journey into opportunity I'm going to be documenting what I've learned, tips, and such on there. I occasionally tweet about investing (stocks/derivatives) and Sweden as well:) If you have questions or want to say hey, feel free to email me at j3elam+hey@gmail.com or dm me at @elamje 4/29/2020 | C# missed the rocket ship Sparked by my curiousity of why SQL Server only has language extension support for Java. I think it has to do with Data Engineering C# missed the big data rocket ship. Do orgs use C# for data workloads? Of course. But It could have become the defacto standard for Big Data had it been less restrictive and platform specific in the early days of explosive data growth. I think it was a mistake that is a similar order or magnitude to missing out on smartphones. There is no doubt that Microsoft had the resources in the early 2000's to create something akin to .NET Core, but for whatever reason, they missed the mark. No cross platform usage meant companies like Google, Twitter, etc. could not use .NET and C# on linux servers and so it struggled to get the same kind of adoption by orgs that operate at that scale. Instead Java and the JVM became king of big data.

1/8/2020 | 1 year on Hacker News You might be wondering, why it matters that I've been on a link aggregator for 1 year. Well, for most people it probably doesn't. However, for me it has led to an inner revival, contract work, learning Clojure, meeting people, understanding leverage, and an awesome new job. Thanks HN. The following thank you is a bit unstructured, but I hope this might resonate with with you. If you don't want to read further, just check out my favorite threads on HN! Awoken geek! The first thing that HN unlocked for me, was the inner nerd that had died the day I believed that the only way to get wealthy was to do finance, medicine or law. As much as I loved my engineering courses in high school and college, at some point I began to think the only way to be successful was to be engaged in those fields, or be an engineering executive. When I found Hacker News, I was delightfully flooded with a wave of people who were obsessed with learning, entrepreneurship, and generally, thought-provoking stuff. Many threads and comments were incredibly nuanced, and some had occasional software or startup celebrity appearances. The contrast between my friends' interests and the online people I found, was large, so it made the discovery of HN that much better. I had learned what its like to have a large group of people gathered based solely on interests. You can make money doing that?! The second thing HN unlocked for me was wonderful short term career prospects. I found people all over the US, and world, working with their technology of choice, making non trivial sums of money doing consulting work. Not to mention the huge corpus of startups and startup employees on HN. To understand why this was news to me, you need some context. To give you a very quick background, I was an undergrad studying computer engineering , torn if I should pursue a career in software, machine learning, or hardware design. Long story short, I got a job working at a large company working on a large enterprise C# app that was very hard to understand. So, after I got my head around web app principles, I quickly realized I was going to have to spend several years there to understand the app and its domain fully, or even have a senior enough position to guide its architecture. When I took the job, I didn't understand that, web software, or what other career paths were possible in software. I wanted fast career growth and fast skill acquisition. About the time I realized this I began seeing startups and contracting as a viable alternative. This realization manifested in seriously reconsidering my current career trajectory. Clojure, so little code, so much power! The third thing HN unlocked for me, was the joy of writing software. Lisp (Clojure specifically) showed me that I didn't need to write a 100 line Java class just to do basic text parsing. This was a wonderful departure from the boring projects I was assigned in college. I realized there are languages without boilerplate. I realized the incredible amount of open-source languages that exist. I realized that dynamic, terse languages like Clojure are a superpower. The most important thing you can do for your business or project is minimize all of the boilerplate and you should really use a language that gets out of your way, and is a pleasure to use, lest your project get buried in the GitHub graveyard. Realizing all of PG's hype about Lisp was rightfully placed was a wonderful feeling. Learning Clojure was the most enlightening thing I have ever done for my software career. No classes? Your code can be manipulated as easily as data? Rick Hickey? Each of these were transformative coming from my C++, C#, and Java background. I realized what it was like to enjoy writing code, and importantly, this sparked motivation and creativity for projects that I would have never set out to do. Leverage is key! The fourth thing HN unlocked for me, was the assurance that I am in the perfect field. I don't know, maybe I'm buying into some VC backed BS, hype train that promises to have a fun career that has high financial reward, but there are very real examples of software people solving problems(typically on the web), with a little bit of code and marketing, that have very real, large economic impact. Back when I was a hardware design intern in college I didn't see any legitimate way to do this inside of the engineering world. Luckily HN has many examples of how this is false. Code is one of the easiest, and most powerful forms of leverage that us software people have at our disposal. I didn't understand this when I first started my big corp job, but it quickly became clear that the successful internet companies and startups all did one thing - write code. At some point the Econ 101 of software clicked, and it was obvious that very few industries that have access to a global, low cost distribution network (internet) and have a cheap army of robots willing to do whatever you code. This is what makes it possible to have monopoly like profit margins, without having a monopoly. Once this clicked, I simply couldn't be content with a salary job that has no equity, slow career growth, and very little creative freedom. This lead to job searching in startup land, building my own MVP, and also moving to contract work. Understanding leverage led me to alter my path to better utilize software and capture value, that I just never would have got working as an enterprise developer. New jobs! The last thing that HN has unlocked for me thus far is contract work and jobs! I am always looking for opportunities to get better at software, but man is it great to get paid to get better. Remember when I said that I wasn't content at my big corp job - well, it turns out the only reason I had the boldness to leave without anything else lined up is because I read a lot of comments and posts from other HNers that took the plunge. Wow, was it exactly what I needed to do! Earlier I mentioned wanting fast career growth and fast skill acquisition; taking two months to build an MVP in a new language was great for that, as was doing contract work with a new technology. To top it off, I was lucky enough for a top-tier C# consulting shop to reach out with an opportunity that I'll be starting tomorrow. Each of these things were a direct result of being engaged on HN! Thanks Hacker News! If you want to say hey, email me at j3elam+hey@gmail.com. If you consider yourself a good writer and have any tips, please email j3elam+writing@gmail.com.

1/2/2020 | Monte Πthon - Monte Carlo, Π, and 8 lines of Python I don't normally talk about math, but I thought this was too magical not to share! If you are like me, you have heard of Monte Carlo simulations and wondered what they are. I don't use them enough to give you a definition, so I will just share an intuitive example of how to use them. I'll start by saying that I am a computer engineer that primarily does web apps, although I'm interested in other spaces. However, being an engineer, developer, or math genius is certainly not required to continue. The aim of this is to show you a simple way to understand and internalize basic monte carlo methods! We are going to calculate π. You can jump to the code, or develop the intuition first. Essentially, we will randomly drop many points into a square with a known area, count how many points also lie inside a circle, then use the ratio - points in circle to total points - and some other information to calculate π. Let's look at the equations we need to do this. The border of circle is made with x-y points that satisfy x^2 + y^2 = r^2, where r is the desired radius. The area of a circle is π * r^2. We will arbitrarily pick r = . 5. Given x-y points, we can find points inside of that circle by checking if x^2 + y^2 < (. 5)^2 radius = . 5 with 10 random x-y points. In the figure above, you can see that there are 6 points in the circle, and 10 points total, so 6 / 10 is the ratio that represents points in circle to points in square. We will multiply the ratio by area(square) to get area(circle), which in this case, is 6 / 10. Recall, this square has an area of 1, so the area(circle) is just equal to the ratio. Since area(circle) = π * r^2, we can rearrange such that π = area(circle) / r^2. In our case π = area(circle) / (. 5)^2. At this point we have two unknowns, π and the area(circle). To solve for π, we need an estimate for the area(circle); if you recall, we just calculated this! It's 6 / 10. And boom! Just like that, we have a method to solve for π. In the picture above, π is (6 / 10) / (. 5)^2, which is 4. To get more accurate, lets write code so the computer can do these steps for us. If you run the code below, you will see that the output gets closer and closer to π's true value(3.141592...) as we increase the number of points. You see that we solved for two unknowns by estimating one of them (area of circle) with random data. As the random data got larger, our estimation got more accurate, and the estimation for π came closer to the truth. I think this is quite magical. Press the green, play button to run for yourself! Code explained: Line 1: import library to generate random x-y points

Line 4: initialize count of points in circle

Line 5: generate random points in 1x1 square

Line 7 & 8: if point is in circle, add to count

Line 9: calculate ratio of points in circle to total points

Line 10: π = circle_area divided by radius squared If you also think this is magical, I highly recommend this short youtube video of a completely unrelated way that π occurs naturally. If you understand why π appears in the Mandelbrot set, please email me at j3elam+pi@gmail.com. If you use monte carlo simulations regularly, I would love to hear how you use them for complex problems. Reach out at the +π email or say hey at j3elam+hey@gmail.com! Submit this with https://www.towardssoftware.com/blog#blog28

John Elam @elamje

I am a consulting software engineer living in Boulder, CO - primarily working with Clojure, Python, .NET & React. I am mainly interested in helping people make stupid simple software, ideally from scratch or very early on (startups and new projects). I also work with mature small & medium sized, non-tech firms to make custom software.

Not available for clojure, python, react, or data engineering projects at the moment. j3elam+consulting@gmail.com

Previously, I worked at PwC, and pre-previously studied Computer Engineering at the University of Texas. Since leaving corporate work I have been making better software, and had time to create several projects which will be linked here in the future. I also run a pearl business.

I am highly interested in high speed rail, hyperloop, and generally, fast transport. Additionally, snowboarding and hiking are a passion. If you are working on anything cool in any of these spaces, I'd love to help out.

See some of my work.

See what I'm up to now.

12/21/2019 | C# to Self Employed Clojure & React Over the past 9 months, I've been somewhat enlightened by a new kind of work that exists for software engineers. I'm going to try to paint a picture of my transition, as well as explain why it could be worth it. You can skip to the pro & con list below, or read the background:-). I'm currently 3 months into an exploratory hiatus from corporate life, and I've been enjoying it massively. About 18 months ago, I was a fresh grad from UT Austin and extremely enthused to start my career in software engineering. I had just gotten a job at PwC, and was somewhat clueless as to what software development was, or could be, as a career. I quickly learned C#, which was my daily language in an enterprise that heavily used the Microsoft stack, and quickly felt stagnation in challenge and learning. It wasn't that things were easy, but for some reason my first year out of the gate was mostly incredibly boring HTML, CSS, and C# copy/paste style coding due to an incredibly mature application (think in the millions of lines of code) that had some hard to grasp domain knowledge caked in. Somewhere in that time I organically stumbled upon Paul Graham, then his essays, and eventually, Hacker News. That orange site profoundly changed the way I saw my career trajectory. Likely, I could, like many older developers around me, stay there for 10-20 years and end up making $200,000+ a year in a cheap, major city, or possibly make partner and double or triple that, but life is too short to wait 10-20 years for an abstract possibility, so I began absorbing all content I possibly could from Hacker News. Three key elements presented themselves to me while browsing - startups, importance of ownership, and fun programming. In startups, I saw the wonderful ability to not deal with typical corporate BS (I loved PwC, but I really don't care to watch 40 hours of accounting ethics videos, annually). In ownership, I saw that really having equity in anything, software related or not, is the only way to build wealth that grows non-linearly with time input (thanks Naval Ravikant). In fun programming I found Clojure. With it I realized I can increase the joy of programming, and decrease boilerplate code by a magnitude simply by using it, rather than Java or C#. (Not to mention the magic of LISP!) Of course, many people don't get wealthy quickly from startups, or just coding. I realized some things in life just take time. With patience in mind, but impatience with my corporate programming growth, I knew I needed something that would be more challenging and provide greenfield projects with architectural decisions. I didn't sense those things would happen quickly at my job, so I spent most of my nights that year learning Clojure, Rust and making a Python web app. Towards the end of the year I decided that I really needed to jump off of the deep end and spend some time trying to make something from scratch, while figuring out what I could do to earn money. The biggest fears were that I wouldn't have healthcare and I would have to watch my savings account deplete. Both of those things will likely be true for you if you do this. Ultimately, I decided it was worth it, so I set out without a clear short-term plan to make money. I had vague ideas that I wanted to sell pearl jewelry, consult in software (thanks patio11), and make a SaaS MVP (read more here). I'm in the middle of all three of those things now, and it's starting to feel normal. I'm working on some contract work I got through Hacker News, making pearl jewelry (eventually want to become an importer), and am basically done with that SaaS MVP. It's certainly more dynamic work, but you might find that it's hard to explain what you do at Thanksgiving and Christmas! I am not sure what the next steps are, but I'm very grateful to PwC for providing a small cushion for this time, and to Hacker News people for catalyzing these realizations. While I have higher ambitions and put much more thought into this than the above paragraphs portray, I want to list some reasons why this work style may, or may not, suit you. There is essentially unlimited schedule flexibility meaning your best hours of the day aren't given to your employer with the tradeoff that you must remain disciplined. There is very high earning potential for starting a business, or consulting as a software developer with the tradeoff of not having steady income or any benefits. There is potential to work on fast paced, dynamic projects using tech that's more interesting to you with the tradeoff that you also might not have work to do for long periods of time. There is potential to work 100 hours a week making every penny of it as a contractor with the tradeoff that you no longer are guaranteed at least 40 hours of work. There is potential to work less with the tradeoff that you likely won't earn more. There is the ability to work for 13 hours straight in mental flow without slack and email pings with the tradeoff that you may feel isolated. There is time to meet with friends and family at any point in the day with the tradeoff that your friends and family might not respect your working hours like you wish. You now will get paid for overtime and deadline pushes with the tradeoff that you no longer get paid to take off for holidays and vacation. Your pay can now be scaled as you please without waiting for an annual review with the tradeoff that you might not get work at your new rate. Your pay is limited by the wider contract market rate, or if lucky, by your niche expertise (with no competitive market rates) with the tradeoff that you have to justify the "Self-employed consultant" slot on your resume to all future employers who worry that you are hiding an unemployed period. While the list above is certainly not exhaustive, these are some of things that crossed my mind when considering if self-employment was a good next step. If you are an ambitious engineer that feels inhibited by the corporate grind, you must determine if you have the power to change your workplace into what you want, or if you need to hit the eject button. If you are interested in more details or just want to chat, say hey at j3elam+hey@gmail.com. To submit this use https://www.towardssoftware.com/blog#blog27

12/17/2019 | Why functional programming

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.

There is an eternal debate around Object Oriented software vs Functional software, and there likely will never be a conclusion, however, this is why I choose Functional when possible.

The API problem is defined as this - for every abstraction and service we create with software, all consumers will have to spend time learning the API. If you are familiar with concept of domain-specific language, or DSL, you can start to understand the trouble here. As nice as an API is, it introduces a new pattern to follow. But, most importantly, it introduces a new pattern to learn. I first heard the term DSL when I was learning Clojure and LISP. Both give you a magical ability (Google-"homoiconic") to create new languages extremely rapidly, but when you create a new language, you make a DSL. While powerful, it's something new that outsiders will have to learn when reading or modifying your code.

The modern web developer knows APIs well. Typically, APIs are used to gain massive leverage and scalability, without having to write much code, or maintain a complex service. As anyone who has used powerful APIs knows, its great once its working, but most of your time is spent looking over the documentation (which sucks most of the time). If you are lucky, you might get it on the first try, but more likely, you have to keep searching through docs, blogs, and stack overflow to find the essence of what you are looking for. The trouble and frustration comes when you've assumed this service is going to plug and play and easy to set up, but you've just spent two hours learning a DSL that you don't really care to learn.

We don't want to learn an API specific pattern, shape, or json language, BUT WE OBJECT ORIENTED PROGRAMmers spend years making complex APIs.

Let me explain - objects have an API in OOP, we don't call it an "API", but that's effectively what the object is. We use an object, we get a contract that says you can call get(), set(), toString(), foo(), bar(), etc. Not only do our objects have an API layer to access functions/methods, but they have an API to access data (I cannot deny functional programming has a data API, also but I will explain why it's better). Back to objects.

Let's look at a class in .NET Core: PolicySchemeHandler. The actual class doesn't matter for our purposes, but if you're interested, it does some authentication related stuff. Now, if you haven't already, please open up that link and tell me which method sounds the most useful. You may have gotten sidetracked by the long method names, or the descriptions that don't actually say what the method does, but rather where it inherits from (sweet, more hyperlinks to follow to figure out how to use this class). This is the API that you are going to spend at least a few minutes learning, if a lot more. It takes almost a minute just to read the class declaration (public class PolicySchemeHandler : Microsoft.AspNetCore.Authentication.SignInAuthenticationHandler ) That's a mouth full!

What I argue, is that this is one class out of hundreds! that your application might need, each with its own interface full of methods. This should not be. Each class you introduce or create for your project brings with it many, many methods that are specific to THAT class alone. Maybe you made a nice interface that required you to implement the sleep() method for the Human, Dog, and Cat classes. Likely, you just typed very similar methods for each one.

So, how does Functional Programming help? I have been spoiled by Clojure, so my opinion will be somewhat colored by my experience with it. In Clojure, we use maps, seqs, vectors, lists, a few other primitive data structures to do all of what a class previously did for us. Something wonderful about this is that I can make 1 function that will accept all of the aforementioned data structures as a parameter and return the correct answer. No need to implement sleep() 3 times (or 100 times for that nasty app you get paid to work on). You implement it once, and it does the magic. You pass in a map with your own shape to represent a human, dog, or cat, and the function treats them all the same.

Would you rather have 1 sleep function for 100 different animal types, or 100 different animal classes with their own sleep method?

This is not a cherry-picked example, this is the norm. You can gain massive leverage by just treating all data as what it is: data! Instead of treating it like an object that is rigid and forces the burden of implementation detail on YOU.

To submit this use https://www.towardssoftware.com/blog#blog26