Building Dynamic Structures for Products
I always liked product price comparison sites, however most of these sites only aggregate products from other shops thereby providing next to no additional value to their users. Examples of this include one-hit wonders such as Pricegrabber.com and Shopping.com.
On the other hand, sites like pricespy.co.uk provide more functionality like their own rating system for shops, product reviews. But their biggest advantage is the advanced search that can query products based on whichever product property the product category supports. They achieve this by documenting product specifications across hundreds of product categories and thousands of products on their site.
I wasn’t satisfied with what was on offer. I wanted to build my own price/product comparison site, but how could I go about doing this?
The heist
First I had to collect a lot of data, which was no easy task. I am sure even PriceSpy realized this early on as in the early versions of the site they allowed users to edit products but since the latest UI update, this feature has been removed. They must now be doing all the documenting inhouse.
Template
From their edit page, it’s apparent that they are using some sort of template structure that supports multiple fields. These fields can have multiple types: free text, selectable items, booleans, and so on. The fields belong to their own categories and can’t be moved, at least not by mere mortals such as myself.
From my understanding, this template gets applied to products and it defines what fields the product supports and in which categories the fields land. This is apparent as a hairdryer does not contain the same GPU Performance field as the console above and can’t be added either.
Requirements
Originally I started my product comparison site with hardcoded structures for products. As I only anticipated the need to support a small niche market that would only need at minimum three, and at most seven, different product types. But even just supporting three hardcoded product types ended up being a nightmare. The amount of code duplication was insane.
This was no way forward. I had to try to make a better system. So I came up with a few rules that my new template system would have to work with after my failure of hardcoded product structures.
- Abstract away all the data into the template that is common across the products.
- All template data should be editable without breaking products on the page. This would mean that I could not use field/category names as keys in my database. As that would break the implementation if I would change their name
- Categories and fields should be movable inside the template, rearrangeable, re-namable, dynamic, again without breaking the products that use these templates.
- Products should inherit the structure from the template.
- Products should get the selectable field values from the template.
- Support for multiple field types.
Field types
These are the following field types I came up with based on my sites requirements, from top:
StringField, UrlField, SingleItemField, MultiItemField.
The first two are free text, and the last two would get their values from the template.
In code:
Structure at last
This would cover the fields but to have the data structured I had to add categories and the Template object itself to tie it all together.
The final template that matches all my requirements looks like this:
The product
As the template itself only contains the field types and selectable value types. The products need to contain actual raw values mapped to the references of the fields they belong to.
An example product for that template looks like this:
This product does not contain any structure, the structure is set by the template. The templateId is set in the template property.
The values array can have raw values in any order as its the template that sets the structure of the final output.
Each key refers to a field reference from the template.
The final output is produced by a method that combines the template and the product. It maps the raw values to the fields that they belong to.
Result
The template system ended up requiring just as much code as the initial three product structures hardcoded. But I have the peace of mind that I only have to maintain one piece of code for all product types and it allows for realtime changes on products. Any change in the template gets reflected on the products instantly. This setup also allows me to query easily based on product specifications.
The only drawback I found was that by not using the field/product names as keys I could not make fancy SEO friendly URLs as the shortIds are not SEO friendly.
Footnote
I have reached out to multiple PriceSpy developers in hopes of getting more information about their setup. But I have not gotten any response.
