Code Generation Post by rambleraptor · Pull Request #7 · rambleraptor/rambleraptor.github.io

rambleraptor · 2021-10-01T00:13:34Z

Adding a post to talk about code generation

eashwar

great article overall! had a few comments here and there

eashwar · 2021-10-01T00:36:07Z

content/posts/codegen.md

+Computers writing code for other computers to run has a very Skynet feel to it
+and sounded too complicated to ever be worth the effort.
+Yet, this practice is so commonplace that there's [built-in
+support](https://docs.bazel.build/versions/main/be/general.html#genrule) for it


CSS nit: you should use the a:hover selector so that the UX for links is better. I almost didn't think that this was a hyperlink.

eashwar · 2021-10-01T00:40:01Z

content/posts/codegen.md

+Quick preface: I'm talking about writing a computer program that outputs
+some kind of file that's funny designed to be read in by a compiler or
+interpreter. I'm not talking about writing data files or schema files. It feels
+natural for computers to be writing those, since they broadly match up with the
+native data types found in programming languages.


I think the grammar here is a little confusing. suggested rewrite for clarity (you don't have to take the way i rewrote it, you can do a different way too):

Suggested change

Quick preface: I'm talking about writing a computer program that outputs

some kind of file that's funny designed to be read in by a compiler or

interpreter. I'm not talking about writing data files or schema files. It feels

natural for computers to be writing those, since they broadly match up with the

native data types found in programming languages.

Quick preface: I'm talking about writing a computer program that outputs

some kind of funny file that's designed to be read by a compiler or

interpreter (not a human). I'm not talking about writing data files or schema files, which

are pretty natural for computers to be writing, since they broadly match up with the

native data types found in programming languages.

eashwar · 2021-10-01T00:40:37Z

content/posts/codegen.md

+
+If you haven't seen GCP lately, there's an unbelievable amount of resource types
+(in the order of hundreds!). Everytime a user wishes to work with a resource of
+a certain type in one of those tools, there has to be code in that cool that


Suggested change

a certain type in one of those tools, there has to be code in that cool that

a certain type in one of those tools, there has to be code in that tool that

eashwar · 2021-10-01T00:43:24Z

content/posts/codegen.md

+resource types and have to make the same fix twelve times. Now, imagine doing
+it a hundred times.
+
+Code generation lets us create a source-of-truth for all this boilerplate.


Really well written section overall -- it almost feels like you're telling a story, the way you're narratively walking the reader through the motivation here.

eashwar · 2021-10-01T00:44:28Z

content/posts/codegen.md

+
+The approach we take is essentially Mad Libs.
+
+<img src="/codegen.png" class="img-fluid hero-image" alt="Codegen example">


Nit: alt text here could be more descriptive for accessibility purposes.

eashwar · 2021-10-01T00:46:25Z

content/posts/codegen.md

+
+## Templated Approach
+
+The approach we take is essentially Mad Libs.


idea: I think it may be useful to start this section off by talking about templating being used with strings in languages, like C# string interpolation or C++ std::format, as these are things many people may have experience with. you could then motivate that you're applying the same principals but at the scale of entire files and more.

eashwar · 2021-10-01T00:49:33Z

content/posts/codegen.md

+function that replicates that template at runtime, and then replace the
+template with a single line of template code. We can more easily test our
+handwritten function. At that point, we just have to make sure that the
+template places the correct values into the function.


Love this section. I think it doesn't feel like a conclusion though, it feels like it abruptly ends here. Might suggest writing some sort of conclusion or something!

eashwar · 2021-10-01T00:49:48Z

content/posts/codegen.md

+discover more places where customizations need to occur, those customizations
+become embedded into our templates. IDEs have poor support for understanding
+template syntax, leaving it up to the team to carefully parse our templates and
+understand what's happening.


glad you discuss the cons to this approach too!

fishythefish

Mostly some quick nitpicks, but I haven't included thoughts on the overall structure or anything. The main thing I noticed was that the post ends rather abruptly.

fishythefish · 2021-10-01T00:38:04Z

content/posts/codegen.md

+in Google's build system.
+
+Quick preface: I'm talking about writing a computer program that outputs
+some kind of file that's funny designed to be read in by a compiler or


Suggested change

some kind of file that's funny designed to be read in by a compiler or

some kind of file that's designed to be read in by a compiler or

fishythefish · 2021-10-01T00:40:17Z

content/posts/codegen.md

+similar. They all follow the same rough logic:
+
+- A user needs a cloud resource to exist in a certain manner
+- The tool calls that cloud resource's GET API to see if the resource exists.


GET/Create/update all have different capitalization, better to be consistent

fishythefish · 2021-10-01T00:41:25Z

content/posts/codegen.md

+
+## How to Generate Code
+
+There's several different approaches to code generation.


Suggested change

There's several different approaches to code generation.

There are several different approaches to code generation.

fishythefish · 2021-10-01T00:42:03Z

content/posts/codegen.md

+  with some amount of this. That's not actually the method I'm going to be
+  talking about. While you don't have to have a compiler background to do this,
+  it does require knowledge of your language's internals. Testing will be more
+  straightforward of any of these.


I don't understand the last sentence.

fishythefish · 2021-10-01T00:43:02Z

content/posts/codegen.md

+infrastructure to create Markdown documentation, Go code, Python code,
+protobufs, YAML files and anything in between. 
+
+The downside to this approach is that the templates can grow unwiedy. As we


Suggested change

The downside to this approach is that the templates can grow unwiedy. As we

The downside to this approach is that the templates can grow unwieldy. As we

fishythefish · 2021-10-01T00:46:08Z

content/posts/codegen.md

+the hunt for more edge cases in generated code, so we can improve the generator
+better.


Suggested change

the hunt for more edge cases in generated code, so we can improve the generator

better.

the hunt for more edge cases in generated code, so we can improve the generator.

fishythefish · 2021-10-01T00:46:25Z

content/posts/codegen.md

+I suspect that this is why using "compiler magic" is so much more tenable for
+teams. It's easier to write unit tests when you have handfuls of functions that
+all alter an AST. That's how developers learn to write code! Write a function
+and then write a text for that function. A template approach means that most of


Suggested change

and then write a text for that function. A template approach means that most of

and then write a test for that function. A template approach means that most of

fishythefish · 2021-10-01T00:48:17Z

content/posts/codegen.md

+Our team does rewrites where we take a large amount of template, write a
+function that replicates that template at runtime, and then replace the
+template with a single line of template code. We can more easily test our


This sentence is kinda "yo dawg, I heard you like templates". Consider restructuring, especially to distinguish the different uses of "template".

fishythefish · 2021-10-01T00:48:46Z

content/posts/codegen.md

+Our team does rewrites where we take a large amount of template, write a
+function that replicates that template at runtime, and then replace the
+template with a single line of template code. We can more easily test our
+handwritten function. At that point, we just have to make sure that the


Suggested change

handwritten function. At that point, we just have to make sure that the

handwritten function than the original template. At that point, we just have to make sure that the

fishythefish · 2021-10-01T00:49:24Z

content/posts/codegen.md

+function that replicates that template at runtime, and then replace the
+template with a single line of template code. We can more easily test our
+handwritten function. At that point, we just have to make sure that the
+template places the correct values into the function.


"template" here is also a little confusing because of the overuse above.

code generation post

d6d56a0

eashwar self-assigned this Oct 1, 2021

eashwar reviewed Oct 1, 2021

View reviewed changes

fishythefish reviewed Oct 1, 2021

View reviewed changes

	a certain type in one of those tools, there has to be code in that cool that
	a certain type in one of those tools, there has to be code in that tool that


		The approach we take is essentially Mad Libs.

		<img src="/codegen.png" class="img-fluid hero-image" alt="Codegen example">


		## Templated Approach

		The approach we take is essentially Mad Libs.

	some kind of file that's funny designed to be read in by a compiler or
	some kind of file that's designed to be read in by a compiler or


		## How to Generate Code

		There's several different approaches to code generation.

	There's several different approaches to code generation.
	There are several different approaches to code generation.

	The downside to this approach is that the templates can grow unwiedy. As we
	The downside to this approach is that the templates can grow unwieldy. As we

		the hunt for more edge cases in generated code, so we can improve the generator
		better.

	and then write a text for that function. A template approach means that most of
	and then write a test for that function. A template approach means that most of

	handwritten function. At that point, we just have to make sure that the
	handwritten function than the original template. At that point, we just have to make sure that the

Conversation

rambleraptor commented Oct 1, 2021

Uh oh!

eashwar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fishythefish left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants