Add Protocol Buffers and gRPC compiler - #21
Draft
OmarAlJarrah wants to merge 4 commits into
Draft
Conversation
Add a compiler that lowers .proto schemas into the spec-agnostic IR, the protobuf/gRPC counterpart to the OpenAPI compiler. It parses a single root .proto with bufbuild/protocompile, resolves well-known-type imports from the parser's bundle, and treats any other unresolved import as a diagnostic (compilers do no file I/O). Coverage spans proto2, proto3, and the 2023 edition: - messages, nested messages, and enums hoist into the flat type registry with IDs derived from their fully-qualified names; field numbers become wire IDs. - presence lowers to Property.Presence (implicit / explicit / required) with proto2 required, proto3 implicit vs optional, and editions features resolved by the parser; protobuf has no null, so nullability stays off. - oneof becomes a WireTagged, exclusive Union reached through one synthetic Flatten wrapper property; synthetic proto3-optional oneofs stay presence markers, not unions. - repeated and map fields hoist List and MapT container nodes; packed vs expanded is recorded on List.Encoding; sint/fixed/zigzag scalar variants lower to Encoding over the base primitive. - enums carry open/closed semantics and allow_alias duplicate values; reserved ranges and names, and custom options, are preserved verbatim in Extensions. - well-known types map faithfully: Timestamp/Duration to date-time primitives, Any/Struct/Value to the schemaless Any node, wrappers to a nullable primitive (or an External under policy), FieldMask/Empty to External. - extension fields attach to the message they extend with Property.ExtensionOf; extension ranges become Model.ExtensionRanges. - services become operation groups; each rpc is an Operation with a gRPC RPCBinding, streaming direction, and idempotency level; an Empty request or response lowers to an absent payload.
…trip Add the test suite for the protobuf compiler, mirroring the OpenAPI compiler: - a conformance corpus of one minimal .proto per protobuf/gRPC capability from the cross-spec matrix (messages, nested types, open/closed enums, oneof, proto3 optional, maps, packed/expanded repeated, scalar wire encodings, proto2 presence and defaults, extensions, reserved, packages, well-known types, all four streaming modes with idempotency, deprecation, doc comments, custom options, and a 2023-edition file), each with a focused assertion and a byte-exact golden IR snapshot; - a full gRPC library service golden snapshot; - a JSON round-trip property test (compile, marshal, unmarshal, deep-equal, and re-marshal for determinism); - registry, option-policy, unresolved-import, and multi-source-rejection tests. Extend the architecture import-graph test so compilers/protobuf may import ir, compilers, protocompile, and the protobuf runtime only.
Simplify the compiler by removing guards that can never fire, so the code matches its actual reachable behavior: - protocompile recovers parser panics internally and returns them as errors, so the compiler's own recover and errParse sentinel were dead; compileRoot now returns the linked descriptor directly and a PanicError flows through as a compile-error diagnostic like any other. - a descriptor's Options() always returns a materialized options message (empty when nothing is set), so the nil-options guards in optionBool, optionsMessage, customOptions, and methodIdempotency were unreachable and are removed. - json.Marshal of the fixed-shape reserved, file-option, and array payloads cannot fail, so those error branches are dropped. - leafType folds MessageKind into its default and syntaxDigit folds proto3 into its default, since those are the only remaining reachable cases.
Extend the corpus and add a white-box helper test so every reachable branch is exercised, matching the repository's 100%-coverage gate: - enrich the corpus with an unmapped well-known type, unsigned/bytes/non-finite proto2 defaults, a repeated encoded scalar, a deprecated oneof member with an explicit json_name, an Empty-input rpc, and an rpc whose only option is deprecated; - add rich-options (message/repeated/map/enum/bytes/unsigned custom options, NaN option values, a group field, and enum-value/service/method options), file-options, and no-package corpus files with assertions and goldens; - add compile tests for a syntax error and an unused-import warning; - add a white-box test covering the pure helpers and the option-rendering recursion-depth guards.
OmarAlJarrah
marked this pull request as draft
July 23, 2026 13:23
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
compilers/protobuf, a compiler that lowers Protocol Buffers.protoschemas into Morphic's spec-agnostic IR — the protobuf/gRPC counterpart to the
existing OpenAPI compiler. Parsing uses
github.com/bufbuild/protocompile, apure-Go
.protocompiler that produces fully linked, feature-resolveddescriptors and bundles the well-known-type imports. Like every compiler, it
does no file I/O: it compiles a single root
.protosupplied as bytes, resolvesgoogle/protobuf/*.protoimports from the parser's bundle, and reports any otherunresolved import as a diagnostic rather than crashing.
It supports proto2, proto3, and the 2023 edition through one lowering, since the
parser resolves edition features into the same descriptor surface the
proto2/proto3 paths expose. The mapping onto the IR:
type registry, with IDs derived from their fully-qualified proto names (the
stable structural identity, never a display name). Field numbers become
Property.WireID; the package becomesNamespace.required/optional, proto3 implicit vsoptional, andeditions
field_presencelower toProperty.Presence(implicit/explicit/required). Protobuf has no null, so
TypeRef.Nullablestaysoff and presence carries the three-state discipline on its own axis.
WireTagged, exclusiveUnionreachedthrough one synthetic
Flattenwrapper property, with each member's fieldnumber preserved as the variant wire ID. Synthetic proto3-
optionaloneofsare presence markers and stay ordinary fields, never unions.
repeatedhoists aList(packed vs expanded onList.Encoding);map<K,V>hoists aMapTwithout exposing the syntheticmap-entry message;
sint/fixed/sfixedlower to anEncodingover the baseprimitive; groups/editions
DELIMITEDcarry thedelimitedencoding.open, per-enum editions feature);
allow_aliasduplicate values survive asdistinct members.
Timestamp/Durationmap to date-time primitives;Any/Struct/Valueto the schemalessAnynode; the wrapper types to anullable primitive by default (or an
Externalunder an injectable policyoption);
FieldMask/EmptytoExternal. Well-known internals are neverhoisted.
extendfields attach to the extendedModelwithProperty.ExtensionOf;extensionsranges becomeModel.ExtensionRanges;reservednumbers/names and custom options (protovalidate and the like) arepreserved verbatim in
Extensions, guarded by the validate pass. Customoptions render deterministically.
servicebecomes anOperationGroup; eachrpcanOperationwith a gRPCRPCBinding, streaming direction (unary, client,server, bidi), and idempotency level. An
Emptyrequest or response lowers toan absent payload.
No IR structs were changed: everything maps onto the existing IR, which already
accounts for wire IDs, oneof, packed repeated, extension ranges, presence, and
editions. One minor, documented degradation: a non-finite proto2 default
(
inf/nan) is dropped with aninfodiagnostic rather than stored as aninvalid decimal, and a
delimitedgroup encoding in a nested (repeated/oneof)position is not re-expressed on the wrapped element.
The compiler is not yet wired into the engine's default registry:
engine.Sniffis YAML/JSON-based and would need a content-detection path for
.proto, which isa separate engine-layer change. The compiler is fully usable through its public
API and registers cleanly into any
compilers.Registry.Test plan
.protoper protobuf/gRPC capability fromthe cross-spec matrix under
testdata/conformance/protobuf/, each with afocused capability assertion plus a byte-exact golden IR snapshot: messages,
nested types, open/closed enums, oneof, proto3 optional, maps, packed/expanded
repeated, scalar wire encodings, proto2 presence and defaults, extensions,
reserved, packages, well-known types, all four streaming modes with
idempotency, deprecation, doc comments, custom options, and a 2023-edition
file.
testdata/golden/protobuf/).cmp.Diff-equal, andre-marshal for determinism.
policy, unresolved-import diagnostics, wrong-options and multi-source rejection.
compilers/protobufmay importir,compilers,protocompile, and the protobuf runtime only.go build ./...,go vet ./...,gofmt -l .(clean),golangci-lint run(clean), and
go test ./...all pass.