Nowadays many e-commerce companies own more than one system for handling their business. This is usually an online store, but also an ERP system for storing the data such as products, customers or invoices. It goes without saying that such data as products’ prices and current supply have to be synchronized. Otherwise, someone might e.g. buy some product which is no longer available. How should such a synchronization be made? In which format should the data be passed between the systems? Which tools and mechanisms should you use not to build everything from scratch?
Data synchronization approaches
The way you synchronize multiple systems depends on business requirements. It is up to how often you want to synchronize data,how many systems you want to sync and whether it should be one- or bi-directional. Below we present two most recommended approaches depending on the above factors.
- One-directional Synchronization via FTP-server for periodical data synchronizations
In case you don’t need to synchronize data real-time you can use a sync via FTP-server. One system (let’s call it system A) should periodically (e.g. every 5 minutes) generate a file with all the needed data and send it to the FTP-server in the network of the second system (system B). The second system should have a background thread periodically checking if any new file is available on FTP. If so, then the file should be imported and the database should be appropriately updated. Thanks to this the whole data is synchronized just by one call to the database, which makes it highly efficient.
- Bidirectional synchronization via WebServices or RESTful services for real-time synchronizations
In case you need to synchronize data real-time you should have a third system that will act as a listener and broadcaster (let’s call this system the Listener). It should expose its API (preferably WebServices or RESTful) for both system A and B and gather all incoming requests. Then it should broadcast the incoming information to the rest of the systems.
In case of an ERP system and online store we can think out of a situation when a customer buys a product via a webshop. The stock of the product is decreased in the eshop’s database. Then the trigger should call a WebService/RESTful service of the Listener to inform him that the stock of the product had been decreased. The Listener received appropriate pieces of info and broadcasts it to the ERP system.
There is also a possibility not to use the Listener and just to make both systems A and B acting as listeners. In this case they should both expose their API’s to make it possible for the second system to inform about any changes.
Data mapping
In case it is enough for you to have synchronization via FTP-server you need to define the file format and scheme that is send to the server and imported by the second system. The most suitable format seems to be XML, but in case of non-complex data (e.g. if you synchronize purely stock data) you can use CSV.
The export of the data from system A to XML/CSV file can be done manually by a programmer, but it might be quite time-consuming. There is in fact no sense to reinvent the wheel as there are many ready solutions you can use. The one we use is the Altova MapForce. This MapForce tools enables you to convert the data from any format to any format by means of a very easy drag-and-drop Map Force designer. The output of MapForce is a project (e.g. in Java or C#) that can be embedded into your application and used for data conversion.
There are many other means of synchronizing multiple systems and data mapping. Though, the ones described above are the most recommended by us. If you know any other better way to sync or map data feel free to share it with us. Leave a comment below!
