Laravel Custom Fields: JSON, EAV Model, or Same Table?

There are situations when we're not sure what the columns of the DB table would be, they need to be flexible. For example, e-shop product properties: size, color, fabric, there may be more in the future. What is the best DB structure? I will show you 3 options and their performance.


Option 1. Properties DB Table and EAV Model

We can define a separate DB table properties and then a Pivot table product_property with many-to-many relations, and an extra field for the value.

This DB structure implements the so-called EAV (Entity-Attribute-Value) Model.

Table "products"

  • id
  • name
  • timestamps

Table "properties"

  • id
  • name (Size, Color, etc)
  • timestamps

Table "product_property"

Schema::create('product_property', function (Blueprint $table) {
$table->foreignId('product_id')->constrained();
$table->foreignId('property_id')->constrained();
$table->string('value');
});

app/Models/Product.php:

class Product extends Model
{
public function properties()
{
return $this->belongsToMany(Property::class)
->withPivot('value');
}
}

Then, if we want to filter the products by specific few property values (like, "Size L and color red"), we can do this.

Controller:

Product::query()
->whereHas('properties', function($query) {
$query->where('properties.name', 'size')
->where('product_property.value', 'L');
})
->whereHas('properties', function($query) {
$query->where('properties.name', 'color')
->where('product_property.value', 'red');
})
->get();

Notice: you may skip the ::query(), I add it personally to have the well-formatted following lines of code, for better readability.

This structure gives you the flexibility to add more properties in the future, without changing the query.

But there's a performance issue, we'll discuss it after we look at the other structure options.


Option 2. JSON Column

We can put all properties just as an unstructured JSON column within the same DB table.

Schema::create('products', function (Blueprint $table) {
$table->id();
$table->string('name');
$table->json('properties');
 
// ...
 
$table->timestamps();
});

Then, we can query the products by property values just like this:

Product::where('properties->size', 'L')
->where('properties->color', 'red')
->get()

Simple, isn't it? But wait until you see the performance results below.


Option 3. More Fields in the Same Table?

Finally, the most straightforward approach: you just add the fields that you know for now, make each of them nullable just in case, and then will add more fields to the same table as they appear.

Yes, it doesn't give you flexibility, but the question is this: do you REALLY need them to be that flexible? In my experience when working with clients, they often demand the system to edit things themselves, and then never really use the functionality.

So, in some cases, it may make sense to "hardcode" things for now, add a few fields when clients ask, and go for the flexibility only when you ACTUALLY need it at least a few times.

With that approach, the structure is just this:

Schema::create('products', function (Blueprint $table) {
$table->id();
$table->string('name');
$table->string('size')->nullable();
$table->string('color')->nullable();
$table->string('fabric')->nullable();
$table->timestamps();
});

The query is also as simple as this:

Product::query()
->where('size', 'L')
->where('color', 'red')
->get();

Performance Comparison

Finally, what is faster, and by how much?

In my benchmark test, I've seeded 1000 products with all three possible structures, and launched this code:

Illuminate\Support\Benchmark::dd([
fn() => Product::whereHas('properties', function($query) {
$query->where('properties.name', 'size')
->where('product_property.value', 'L');
})->whereHas('properties', function($query) {
$query->where('properties.name', 'color')
->where('product_property.value', 'red');
})
->get(),
fn() => Product::query()
->where('properties->size', 'L')
->where('properties->color', 'red')
->get(),
fn() => Product::query()
->where('size', 'L')
->where('color', 'red')
->get(),
], 5);

It measured all three queries 5 times, and here are the average results:

array:3 [
0 => "6.688ms"
1 => "1.911ms"
2 => "1.612ms"
]

I tried a few more times, but the results were pretty much the same:

  • whereHas EAV approach is 3-4x slower than other structures
  • JSON approach is only 10-30% slower than the approach of regular columns

Not sure if you would draw the same conclusion from my benchmark, or maybe some queries may be optimized.

All in all, my suggestion would be to avoid the EAV model unless you REALLY need it. JSON columns are okay-ish in terms of performance, but depending on your (more complex?) structure may also be slower or less convenient to query.

The good old approach of just adding more fields to the same table may not sound "sexy" but if you go for simplicity and the quickest performance "for now", you should consider starting with that structure.

avatar

I had to select one of the three options several times, and I had never thought about performance, only scalability, depending on whether it is a large or small app.

avatar

6 miliseconds is indeed 3-4x slower than 1.6 ms, but honestly... who cares? Most of the webshops works with fewer than 10 000 products. And if I multiply this times with 10, we are very good too.

We are takling about miliseconds!

6 or 60... humans cannot make difference. and you kill the flexibility of your app.

avatar

My example is very simplified, with much fewer DB columns than real eshops, and with very simplified query itself that may have more conditions.

So it's not about miliseconds, it's about the approach and the main idea for best performance practices.

But you're right, if in your cases you wouldn't FEEL the difference, then no big deal. Then, if your shop grows enough, you may refactor later.

avatar

I have been using this approach on many application that need such flexibility. However, it would be great if such support was available in Quick Admin Panel :-)

avatar

Sorry, at the moment it's not planned in Quick Admin Panel :)

avatar

Allow me a php not Laravel comment, because it has caused me much pain multiple times already: Wordpress is version 1) throughout, and Woocommerce on top of it increases the complexity of your example, because all blog posts, products, orders etc. are in the same table (wp_posts), with the wp_post_meta table holding the different properties. So a customer order of 5 products (that themselves have variations) becomes a SQL nightmare. And to the comment above "it's only milliseconds": exactly this brings web shop databases to their knees, when a few hundred users access it at the same time, and the database must dig through millions of records for each.

👍 1
avatar

And thanks for doing the benchmark work, Povilas. Wouldn't have thought that the Json solution is so close to the native one.

avatar

Thanks for the comment! And again, it's only MY benchmark in certain situation, other situations may bring different results.

avatar

I think the main problem in Wordpress is storing all that kind of data in one table. That's why in the latest Woocommerce versions you can have separate tables for customers,orders,products etc.

avatar

First time I hear about this @vilyo Do you have a link to that information? I would like to learn more about it

avatar

I don't know if it's ethic to post such links here. Search for Woocommerce 7.1 and High Performance Order Storage (HPOS). Now it's in woocommerce as an option and have to be enabled. Sooner or later will be in the core by default.

avatar

Thanks a lot, I will check that out carefully

avatar

It's interesting how this variant for custom fields that I am using will be compared to those in the article. So, I am using ModelMeta for the models I need, for example - User=>UserMeta, Customer=>CustomerMeta .. I have user_id/customer_id, key and value field in the Meta tables. What are the pluses and minuses related to having property and user_properties tables?

avatar

The plus is more flexibility, I guess, for anyone to see the list of properties and add more properties to that list (like, non-dev administrator). In your case, it's hard/impossible to see what are the possible keys, if they are not already in the table.

avatar

For EAV, the problem of multiple whereHas can be solved by a single join, which will increase its performance

Instead of:

value = [ 1 => [1,2,3], 2 => [4,5,6], ];

foreach ($value as $option_id => $option_value_ids) { $builder->whereHas('option_values', function (Builder $b) use ($option_value_ids) { $b->whereIn('option_value_id', $option_value_ids); }); }

run:

$builder->whereHas('option_values', function (Builder $b) use ($value) { $b ->select('product_type_id') ->selectRaw('COUNT(DISTINCT(option_id)) as counter') ->whereIn('optionvalue_id', Arr::flatten($value)) ->groupBy('product_type_id') ->having('counter', count($value)); });

Like our articles?

Become a Premium Member for $129/year or $29/month
What else you will get:
  • 68 courses (1188 lessons, total 43 h 18 min)
  • 90 long-form tutorials (one new every week)
  • access to project repositories
  • access to private Discord

Recent New Courses