The Art of Receiving Data from Backend, for Frontend Developers Who Want to Sleep Peacefully

Abstract

Data obtained when fetching from an API always has type any. We never really know what form the data takes. Therefore, when receiving data, we must ensure that it matches what we expect.

If the data differs from our expectations, it should throw an error, not just run normally as if nothing happened.

zod is the library used for data validation in this article.

Try it directly at zod-undefined-playground.vercel.app

Zakiego

@zakiego

·Follow

Kadang mikir, i'm too strict harus diparse pakai Zod (@zodtypes), tapi tetap bertahan, untuk tidur yang lebih nyenyak Return dari fetch itu 'any'. Once your backend ngubah API, ga akan ketahuan errornya. Tapi kalau diparse pakai Zod, detik itu juga ditampar sama error di console

˗ˏˋ daisy its sisfor 26

@cinnamo0ns_

ya Allah 1 jam debugging ternyata kurang .data doang

6:06 AM · Nov 30, 2023

198

Read 15 replies

Introduction

When working in a team with medium to high complexity, there are at least two roles: Frontend Engineer and Backend Engineer. Simply put, Frontend is someone who creates web interfaces to look beautiful, while Backend is the person behind the scenes who manages data flow in and out of the database.

The most common communication channel between these two roles is usually through APIs.

A Backend developer will provide an endpoint, for example, dummyjson.com/products. Then, the Frontend will fetch to that endpoint:

const resp = await fetch("https://dummyjson.com/products").then((res) =>
  res.json(),
);

console.log(resp);

Uncertainty

We live in a world full of uncertainty. And the only certain thing in this world is uncertainty itself.

The same applies to Frontend developers.

For example, when fetching from the API above, we get a response like this:

{
  "products": [
    {
      "id": 1,
      "title": "iPhone 9",
      "description": "An apple mobile which is nothing like apple",
      "cost": 549,
      "discountPercentage": 12.96,
      "rating": 4.69,
      "stock": 94,
      "brand": "Apple",
      "category": "smartphones",
      "thumbnail": "https://cdn.dummyjson.com/product-images/1/thumbnail.jpg",
      "images": [
        "https://cdn.dummyjson.com/product-images/1/1.jpg",
        "https://cdn.dummyjson.com/product-images/1/2.jpg",
        "https://cdn.dummyjson.com/product-images/1/3.jpg",
        "https://cdn.dummyjson.com/product-images/1/4.jpg",
        "https://cdn.dummyjson.com/product-images/1/thumbnail.jpg"
      ]
    },
    {
      "id": 2,
      "title": "iPhone X",
      "description": "SIM-Free, Model A19211 6.5-inch Super Retina HD display with OLED technology A12 Bionic chip with ...",
      "cost": 899,
      "discountPercentage": 17.94,
      "rating": 4.44,
      "stock": 34,
      "brand": "Apple",
      "category": "smartphones",
      "thumbnail": "https://cdn.dummyjson.com/product-images/2/thumbnail.jpg",
      "images": [
        "https://cdn.dummyjson.com/product-images/2/1.jpg",
        "https://cdn.dummyjson.com/product-images/2/2.jpg",
        "https://cdn.dummyjson.com/product-images/2/3.jpg",
        "https://cdn.dummyjson.com/product-images/2/thumbnail.jpg"
      ]
    }
  ]
}

The data received is product data in the form of an array, containing detail objects for each product. Then we render this data to each component for display. There's product name, price, rating, image, etc. Everything runs smoothly.

Try it through this playground zod-undefined-playground.vercel.app

However, on a day and moment we don't know about, the Backend changes the format of the data provided.

The data becomes:

{
  "products": [
    {
      "id": 1,
      "title": "iPhone 9",
      "description": "An apple mobile which is nothing like apple",
      "price": 549,
      "discountPercentage": 12.96,
      "rating": 4.69,
      "stock": 94,
      "brand": "Apple",
      "category": "smartphones",
      "thumbnail": "https://cdn.dummyjson.com/product-images/1/thumbnail.jpg",
      "images": [
        "https://cdn.dummyjson.com/product-images/1/1.jpg",
        "https://cdn.dummyjson.com/product-images/1/2.jpg",
        "https://cdn.dummyjson.com/product-images/1/3.jpg",
        "https://cdn.dummyjson.com/product-images/1/4.jpg",
        "https://cdn.dummyjson.com/product-images/1/thumbnail.jpg"
      ]
    },
    {
      "id": 2,
      "title": "iPhone X",
      "description": "SIM-Free, Model A19211 6.5-inch Super Retina HD display with OLED technology A12 Bionic chip with ...",
      "price": 899,
      "discountPercentage": 17.94,
      "rating": 4.44,
      "stock": 34,
      "brand": "Apple",
      "category": "smartphones",
      "thumbnail": "https://cdn.dummyjson.com/product-images/2/thumbnail.jpg",
      "images": [
        "https://cdn.dummyjson.com/product-images/2/1.jpg",
        "https://cdn.dummyjson.com/product-images/2/2.jpg",
        "https://cdn.dummyjson.com/product-images/2/3.jpg",
        "https://cdn.dummyjson.com/product-images/2/thumbnail.jpg"
      ]
    }
  ]
}

What changed? At first glance, nothing seems to have changed. Everything looks the same. But when you look more carefully, you realize that one key has changed from cost to price.

Let's look at the code below. What happens when we try to print cost (should be price).

const resp = await fetch("https://dummyjson.com/products").then((res) =>
  res.json(),
);

// should be `price`, not `cost`
console.log(resp.products[0].cost);
// undefined

The result is just undefined, not an error.

And when displayed on a web page, everything runs smoothly, no errors. But the product price becomes empty.

Try it through this playground zod-undefined-playground.vercel.app/without-zod

So what's the problem? The problems are:

If this change made by the Backend happens without us knowing. We don't get notified that there was a change in the API response format. As a result, the logic of our application will run outside our expectations.
Because the return is only undefined, not an error, we won't get an error notification. This means the application continues to run as usual, but actually something wrong is happening. For example, the price that should be displayed becomes empty due to the wrong key. No error, just empty. We won't know this until we find it ourselves, or someone reports it.

Safeguard

Allow me to introduce zod (zod.dev).

With the same API as before, the received response data is first validated against the data schema we want. After that, the data is used.

import { z } from "zod";

// response from fetch always has type any
const resp = await fetch("https://dummyjson.com/products").then((res) =>
  res.json(),
);

// define the desired data schema
const schema = z.object({
  products: z.array(
    z.object({
      id: z.number(),
      title: z.string(),
      description: z.string(),
      cost: z.number(), // <--- still using `cost`, not `price`
      discountPercentage: z.number(),
      rating: z.number(),
      stock: z.number(),
      brand: z.string(),
      category: z.string(),
      thumbnail: z.string(),
      images: z.array(z.string()),
    }),
  ),
});

// data is parsed using the defined schema
const data = schema.parse(resp);

console.log(data);

The zod schema above assumes the price key still uses cost, not price.

See the result when executed.

ZodError: [
  {
    "code": "invalid_type",
    "expected": "number",
    "received": "undefined",
    "path": [
      "products",
      0,
      "cost"
    ],
    "message": "Required"
  }
]

And... error. That's what should happen.

Try it through this playground zod-undefined-playground.vercel.app/with-zod

The error occurs because the schema we defined requires a cost key with a number type value. But what we got was a cost value with undefined type.

When the received data doesn't match our expectations, it should throw an error. The goal is so we can fix it as soon as possible.

What is Zod?

It seems we've gone too far ahead, let's take a few steps back to understand what zod actually is.

Zod is a TypeScript-first schema declaration and validation library.

From the definition above, zod is a library for defining schemas and validating data. Data validation is needed to maintain and ensure data consistency, so programmers can sleep peacefully without worrying about data suddenly changing without notice.

Let's try it, slowly.

First, install zod in our project.

pnpm add zod

Then, import and create a simple schema.

We create a schema called rankSchema that requires receiving number input. However, instead of entering a number, we try entering a string.

See the result.

import { z } from "zod";

const rankSchema = z.number(); // schema only accepts number input

const rank = rankSchema.parse("2"); // we try entering a string

console.log(rank);

// ZodError: [
//   {
//     code: "invalid_type",
//     expected: "number",
//     received: "string",
//     path: [],
//     message: "Expected number, received string",
//   },
// ];

The result is an error. Because we force only true number input to be allowed, but the input received was a string.

If the input is actually a number, there will be no error.

import { z } from "zod";

const rankSchema = z.number();

const rank = rankSchema.parse(1);

console.log(rank);
// 1

Let's continue to create a more complex schema.

We define a user schema called userSchema, the input data must be an object with at least name and age keys. Then we enter input data according to that schema.

import { z } from "zod";

const userSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const input = {
  name: "Zaki",
  age: 20,
};

const user = userSchema.parse(input);

console.log(user);
// {
//  name: "Zaki",
//  age: 20,
// }

The result is everything runs safely because the schema and input match. Good.

Then let's try what happens if the input doesn't match. The age key is removed from the object.

import { z } from "zod";

const userSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const input = {
  name: "Zaki", // <--- age removed
};

const user = userSchema.parse(input);

console.log(user);

// ZodError: [
//   {
//     code: "invalid_type",
//     expected: "number",
//     received: "undefined",
//     path: ["age"],
//     message: "Required",
//   },
// ];

Because we removed age, while in the schema age is required, an error appears.

Schema Options

There are many type options that can be used to define schemas. Here are some examples:

// primitive values
z.string();
z.number();
z.bigint();
z.boolean();
z.date();
z.symbol();

// empty types
z.undefined();
z.null();
z.void(); // accepts undefined

// catch-all types
// allows any value
z.any();
z.unknown();

// never type
// allows no values
z.never();

z.enum(["Salmon", "Tuna", "Trout"]);

Complete documentation can be found at zod.dev.

Not all values are required, we can make them optional, for example z.string().optional().

Or if we don't know (or are too lazy to define) we can use z.any().

Type Inference

Another benefit of using schema validation is that we can also get the Type from the defined schema.

import { z } from "zod";

const userSchema = z.object({
  name: z.string(),
  age: z.number(),
});

type User = z.infer<typeof userSchema>;
//   ^ type User = { name: string; age: number; }

So, if we need the User type as a props type in a component, we just need to export it.

Conclusion

Data obtained from fetch always has value any. We never really know what form the data takes. It's like buying a cat in a bag - the contents could actually be a crocodile.

Therefore, it's necessary to validate the data we receive.

So we can sleep more peacefully.

Yehez | Kang Ketik

@YehezGun

·Follow

"Return dari fetch itu 'any'" Dari statement inilah kultum @__r17x & @mustafasegf mulai mendoktrin gw😆 Itu gw inget betul dri jam 2 siang ampe jam 5 sore, full berdiri di cafe ngobrolin tipe data yg berujung pada konsep Monad -> OCaml -> Rust😆

Zakiego

@zakiego

6:46 AM · Nov 30, 2023

Read 3 replies

mustafa

@mustafasegf

·Follow

yep, semua io itu perlu di parse dulu bentuk dan tipe nya menjadi bentuk yg kita inginkan

7:00 AM · Nov 30, 2023

Read 1 reply

Resources

Here are some resources relevant to this article:

Alternatives

zod is not the only schema declaration and validation library.

There are other alternatives like valibot and arktype.

MZP

@Maz_Ipan

·Follow

Replying to @tibudiyanto

Menurutku, Zod lebih battle-tested, lebih banyak pengguna, lebih populer, lebih aktif, lebih banyak maintainer. Tapi untuk projek yang pure frontend (bukan kode untuk server side), approach-nya Zod bisa bikin bengkak JS bundl, meskipun yang dipakai hanya 1 atau 2 fiturnya.

12:25 PM · Feb 21, 2024

Read 1 reply