# How to evolve an Avro event schema without breaking consumers > Register an Avro schema for a topic, set BACKWARD compatibility, and ship a field with a default that consumers you have not redeployed keep reading. Source: https://voxgig.com/howto/evolve-event-schemas-with-a-schema-registry - Audience: api-producer - Level: intermediate - Languages: typescript, javascript - Verified: 2026-10-21 - Published: 2026-10-17 ## Short answer Register the topic's Avro schema in a schema registry under a BACKWARD-compatible subject, then add fields only with a default, so a consumer holding the previous schema still resolves every record. Each record carries a zero byte, the schema id, and the Avro bytes; the consumer fetches the writer schema by id and resolves it against its own. The registry refuses a field without a default at registration, which is where the break should surface. --- ## You will need Node 22 or later, a Kafka topic whose records are Avro, and a schema registry that speaks the Confluent REST API. Verified 2026-10-21 against Node 22.22.0, `@kafkajs/confluent-schema-registry` 4.1.0, and `avsc` 5.7.9, installed from npm in the sample directory. The sample runs no broker and no real registry. An in-process stand-in answers the registry routes the client calls, with the compatibility check done by Avro's resolution rules as `avsc` implements them. The record values are the buffers a producer would send. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [Apicurio Registry with JSON Schema](https://www.apicur.io/registry/docs/apicurio-registry/3.4.x/getting-started/assembly-registry-compatibility-modes.html) | You run your own registry and want one place for Avro, JSON Schema, Protobuf, OpenAPI, and AsyncAPI artifacts | A service you operate, and JSON Schema compatibility judged on a constraint model that is harder to reason about than Avro's field rules | Your clients are the Confluent serializers and you would rather not test them against a second API | | [AsyncAPI documents checked in CI](https://www.asyncapi.com/docs/tools/cli/usage) | The contract is reviewed in pull requests and no runtime dependency is wanted | Nothing at run time compares the bytes with the document, so a producer can ship what the document forbids | Several producers, or consumers you do not deploy yourself | | [AWS Glue Schema Registry](https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html) | Producers and consumers on AWS, with MSK, and no registry to run | JSON Schema only up to draft-07, its own wire format and serializer libraries, and mode names that differ from the Confluent ones | Clients outside the AWS SDKs, or a wire format other tools already read | | [Confluent Schema Registry with Avro](https://docs.confluent.io/platform/current/schema-registry/index.html) | Kafka with clients in several languages, all of which read the same five-byte prefix | A service with its own storage topic to run, subject names that are hard to change later, and a default check against the latest version only | JSON on the wire that people read by hand, or no Kafka at all | | [Version field in the payload, no registry](https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md) | One producer, a few consumers you deploy, and JSON payloads | A version switch in every consumer, with no check that refuses an incompatible change before it ships | More than a handful of consumers, or any consumer you do not deploy | The registry buys one thing: a change that would break a reader is refused before a producer ships it. The Confluent registry is the one most clients already speak, at the cost of a service and a subject layout you choose once. Apicurio takes more formats behind a compatible API, for a second service to run. AsyncAPI in CI adds no runtime dependency, and for the same reason it never sees the bytes. A version field costs nothing at all and moves the whole burden into every consumer. ## Name the subject and register the first schema A subject is the name the registry keeps versions under, and the compatibility check runs within a subject. The Confluent serializers derive it from the topic, the record, or both, by a [subject name strategy](https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/index.html#subject-name-strategy). The client here takes the subject as a string, so the sample makes the choice in one module. ```js title="subjects.mjs" export const strategies = { // One schema per topic. The default, and right until a topic carries two event types. TopicNameStrategy: (topic, schema, { isKey = false } = {}) => `${topic}-${isKey ? 'key' : 'value'}`, // One schema per record type, whatever topic it travels on. RecordNameStrategy: (topic, schema) => fullName(schema), // One schema per record type per topic. TopicRecordNameStrategy: (topic, schema) => `${topic}-${fullName(schema)}`, } ``` The producer registers the schema once and gets an id. `encode` then writes the record value in the registry's [wire format](https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/index.html#wire-format): a magic byte of zero, the schema id as a four-byte big-endian integer, and the Avro binary.[^1] ```js title="register.mjs" const subject = strategies.TopicNameStrategy(topic, orderPlacedV1) const { id } = await registry.register({ type: SchemaType.AVRO, schema: JSON.stringify(orderPlacedV1) }, { subject }) console.log(`\nregistered shop.orders.OrderPlaced under ${subject}: id ${id}, compatibility ${local.levelOf(subject)}`) // encode() writes the wire format: one zero byte, the schema id as a big-endian int32, then the // Avro binary. This buffer is what goes in the Kafka record's value. const value = await registry.encode(id, { orderId: 'ord_1', customerId: 'cus_9', totalCents: 4200 }) ``` ```bash node register.mjs ``` ```text output TopicNameStrategy orders-value RecordNameStrategy shop.orders.OrderPlaced TopicRecordNameStrategy orders-shop.orders.OrderPlaced registered shop.orders.OrderPlaced under orders-value: id 1, compatibility BACKWARD record value: 19 bytes byte 0 magic byte 0 bytes 1-4 schema id 1 (0 0 0 1) bytes 5-18 avro 0a6f72645f310a6375735f39d041 decoded by a fresh consumer: {"orderId":"ord_1","customerId":"cus_9","totalCents":4200} registry requests: GET /config/orders-value POST /subjects/orders-value/versions PUT /config/orders-value GET /schemas/ids/1 ``` Fourteen of the nineteen bytes are the order. The other five are what lets a consumer that has never seen this schema read it: it takes the id from the prefix, fetches the writer schema with `GET /schemas/ids/1` once, and caches it. The `PUT /config` is the client setting its default level, BACKWARD, which is also the [registry's default](https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html#compatibility-types), on a subject that had none. Pick the strategy before the first producer ships, because changing a subject later means re-registering history. Topic name is right while a topic carries one event type; several types need a record-based strategy, as the last section shows. ## Add the field under BACKWARD BACKWARD means that a consumer using the new schema can read data produced with the last schema, in the words of the Confluent documentation. The check is Avro's own [schema resolution](https://avro.apache.org/docs/1.12.0/specification/#schema-resolution). Fields are matched by name, and a writer's field the reader lacks is skipped. A reader's field the writer lacks is filled from the reader's default or, with no default, is an error. So a field arrives with a default, and a consumer on the previous schema keeps resolving every record. ```js title="evolve.mjs" // The change: a field with a default, registered under BACKWARD. The registry checks that the // new schema can read what v1 wrote, and only then hands out an id. const { id: id2 } = await producer.register(asAvro(orderPlacedV2), { subject, compatibility: 'BACKWARD' }) const newBytes = await producer.encode(id2, { orderId: 'ord_2', customerId: 'cus_9', totalCents: 1500, currency: 'USD' }) console.log(`v1 is id ${id1}, v2 is id ${id2}, subject ${subject} is ${local.levelOf(subject)}`) // The consumer that has not been redeployed still holds v1 as its reader schema. Resolution // matches fields by name, so the field it does not know is skipped. const oldConsumer = new SchemaRegistry({ host: local.url }) const readByOld = await oldConsumer.decode(newBytes, { [SchemaType.AVRO]: { readerSchema: orderPlacedV1 } }) console.log(`old consumer, reader v1, new bytes: ${JSON.stringify(readByOld)}`) // The redeployed consumer holds v2 and rewinds to the start of the topic. The field v1 never // wrote is filled from the default in its own schema. const newConsumer = new SchemaRegistry({ host: local.url }) const readByNew = await newConsumer.decode(oldBytes, { [SchemaType.AVRO]: { readerSchema: orderPlacedV2 } }) console.log(`new consumer, reader v2, old bytes: ${JSON.stringify(readByNew)}`) ``` ```bash node evolve.mjs ``` ```text output v1 is id 1, v2 is id 2, subject orders-value is BACKWARD old consumer, reader v1, new bytes: {"orderId":"ord_2","customerId":"cus_9","totalCents":1500} new consumer, reader v2, old bytes: {"orderId":"ord_1","customerId":"cus_9","totalCents":4200,"currency":"EUR"} register v3 (channel, no default): HTTP 409 Confluent_Schema_Registry - Schema being registered is incompatible with an earlier schema for subject "orders-value", details: no matching field for default-less shop.orders.OrderPlaced.channel versions of orders-value: [{"id":1,"version":1},{"id":2,"version":2}] ``` The consumer on v1 reads a v2 record and never sees `currency`. The consumer on v2 reads a v1 record and gets `EUR`, the default, for a field the producer never wrote.[^2] The reader schema is passed to `decode`, and the writer schema comes from the registry by id, which is the whole mechanism. Then v3 adds `channel` with no default, and the registration is answered with 409, the status the [REST API](https://docs.confluent.io/platform/current/schema-registry/develop/api.html) documents for an incompatible schema, before any producer can use it. The subject stays at two versions. Each format has its own rules. Avro's are the resolution rules quoted earlier, which is why the check is a short list. The Confluent [compatibility guide](https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html) calls the Protobuf and JSON Schema rules more nuanced, and recommends BACKWARD_TRANSITIVE for Protobuf because adding a message type is not forward compatible. The same two calls exist in the other clients. In [confluent-kafka-python](https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html) the `AvroDeserializer` takes the reader schema as `schema_str`, and in Go the [franz-go `sr` package](https://pkg.go.dev/github.com/twmb/franz-go/pkg/sr) pairs a `Serde` with a `ConfluentHeader` that produces the same five-byte prefix. ## Check it worked The test that matters registers v2 and then v3 under BACKWARD, and asserts that the second attempt is refused with a 409 that names the field. ```js title="evolve.test.mjs" test('under BACKWARD a field with a default registers and a field without one is refused', async () => { const registry = new SchemaRegistry({ host: local.url }) const { id: id1 } = await registry.register(asAvro(orderPlacedV1), { subject: 'backward-value' }) const { id: id2 } = await registry.register(asAvro(orderPlacedV2), { subject: 'backward-value', compatibility: 'BACKWARD' }) assert.notEqual(id1, id2) await assert.rejects(registry.register(asAvro(orderPlacedV3NoDefault), { subject: 'backward-value' }), (err) => { assert.equal(err.status, 409) assert.match(err.message, /incompatible with an earlier schema/) assert.match(err.message, /default-less shop\.orders\.OrderPlaced\.channel/) return true }) assert.deepEqual(local.versions('backward-value').map((v) => v.version), [1, 2]) }) ``` ```bash node --test evolve.test.mjs ``` ```text output 1..7 # tests 7 # suites 0 # pass 7 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 304.828737 ``` Seven tests pass. The others pin the five-byte prefix, both reader schemas, and the subject names each strategy produces. One pins the difference between the levels: removing a field passes BACKWARD and fails FORWARD and FULL, since a consumer on the previous schema could not fill it. Another pins the transitive case. ## When it goes wrong The unit tests are green and the registry says no. A test that encodes and decodes with one schema proves that schema agrees with itself, and the registry compares it with what is already registered. The compatibility endpoint answers the same question without registering anything. ```bash node pitfall.mjs ``` ```text output v3 round trip in a unit test: {"orderId":"ord_3","customerId":"cus_9","totalCents":10,"currency":"EUR","channel":"web"} v3 against orders-value: {"is_compatible":false,"messages":["no matching field for default-less shop.orders.OrderPlaced.channel"]} avsc fromBuffer on the framed value: throws "trailing data" avsc fromBuffer after 5 bytes: {"orderId":"ord_1","customerId":"cus_9","totalCents":4200} OrderCancelled under orders-value: HTTP 409 Confluent_Schema_Registry - Schema being registered is incompatible with an earlier schema for subject "orders-value", details: no alias found for shop.orders.OrderPlaced OrderCancelled under shop.orders.OrderCancelled: id 2 ``` Run the compatibility check in CI with `verbose=true`, so a pull request that adds a field without a default fails there rather than in a deploy. A plain Avro decoder throws on every record. It was handed the framed value, so it read the magic byte as the first field. Here the result is `trailing data`; another schema could yield wrong values and no error. Strip the five bytes, or use a registry-aware deserializer, which reads the id and fetches the schema. A second event type on the topic is refused as incompatible. Under the topic name strategy, both records share the subject `orders-value`, and `OrderCancelled` cannot read what `OrderPlaced` wrote. The record name strategy gives each type its own subject, and the registration goes through. ## When not to do this Do not add a registry to a topic with one producer and one consumer in one repository. A shared schema file and the same compatibility check run in that repository's tests gives the same protection without a service to run. Do not set a subject to NONE to get a deploy out. The check that refused the change is the one thing the registry does, and a consumer that cannot read the topic is the outage you bought the registry to prevent. Do not choose FORWARD because the producer team wants to remove fields. BACKWARD is the Confluent default so that a consumer can be rewound to the start of the topic, and FORWARD gives up that guarantee.[^3] Do not read BACKWARD as a check against the whole history. It compares the candidate with the latest version only, so a version that drops a default the previous version added passes, though older records become unreadable. BACKWARD_TRANSITIVE checks every earlier version, at the cost of refusing changes the registry would otherwise allow.[^4] ## Related how-tos - [Describe a Kafka topic with AsyncAPI](/howto/describe-a-kafka-topic-with-asyncapi) - [Make queue consumer writes safe under redelivery](/howto/queue-consumer-writes-under-at-least-once) - [Define a gRPC service in a proto3 file](/howto/define-a-grpc-service-in-proto3) - [Evolve an error contract without breaking clients](/howto/evolve-an-error-contract-without-breaking-clients) ## Last verified Verified 2026-10-21 against Node 22.22.0, `@kafkajs/confluent-schema-registry` 4.1.0, and `avsc` 5.7.9. Every output block is what the command preceding it printed. The registry in the sample is an in-process stand-in for the routes the client calls, with compatibility judged by the schema resolution in `avsc`. Its 409 is the status the API reference documents for an incompatible schema, with a message of the stand-in's own. Confluent Schema Registry, Apicurio Registry, and the Glue registry were not run; their behavior is taken from their documentation. No broker was involved, so the record values are the buffers a producer would send. [^1]: The [wire format table](https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/index.html#wire-format) calls byte 0 the version byte and gives its value as "0 when using the schema ID (the default)," a parenthesis that does some work. The same page documents a second layout, in which the version byte is 1 and bytes 1 to 16 carry a schema GUID. So the magic byte is a version number that has acquired a second value, and a decoder that checks for zero is checking something after all. [^2]: Avro's [specification](https://avro.apache.org/docs/1.12.0/specification/#schema-declaration) is careful about what a default is for. A default is "only used when reading instances that lack the field," and its presence "does not make the field optional at encoding time," as the specification puts it. A producer on v2 still has to supply `currency` on every record. The default exists for the reader, and for the records written before anyone thought of the field. [^3]: The Confluent [compatibility page](https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html#compatibility-types) gives the reason BACKWARD is preferred for Kafka: "so that you can rewind consumers to the beginning of the topic," a rewind FORWARD does not guarantee. The same page adds that FORWARD is harder to work with, because "you need to anticipate all future changes," which is a fair description of most forward planning. [^4]: The transitive levels have two names. Confluent and [Apicurio](https://www.apicur.io/registry/docs/apicurio-registry/3.4.x/getting-started/assembly-registry-compatibility-modes.html) write BACKWARD_TRANSITIVE, and [AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html) writes BACKWARD_ALL for a check it defines the same way, against all previous schema versions. Apicurio's documentation supplies the case the plain level misses: a third version that passes the BACKWARD check against the second and fails the BACKWARD_TRANSITIVE check against the first.