What’s data transfer?
In essence, transferring involves exporting an entity, such as a database or dashboard, from one server as a dump file and creating a new entity by uploading this file to another server.
Depending on the chosen export method, the dump file might "remember" its origin, which allows maintain synchronization between entities on the target servers and the original entity.
Keep in mind that synchronized entities on the target server have restricted editing options to ensure consistency with their source.
User accounts on both source and target servers are independent. One user can download the entity, while another user uploads it to the target server.
To utilize this functionality, user accounts on both sides should have the Syndicate Manager role and the appropriate Creator role corresponding to the entity type (e.g., Dashboard Creator for dashboards, Database Creator for databases, etc.).
Why you may need it
Though it is possible to dump and restore entities within the same server, we suggest using the standard cloning and sharing functionality for a more streamlined approach to managing access to your entities on the same instance of DataTile.
Cross-server data transfer is advantageous in numerous scenarios where production infrastructure and research environments are distinct. Separation may arise due to staged quality checks, security restrictions, or business-specific needs.
Below are two primary use cases.
Syndicated studies
Effortlessly distribute syndicated databases to customers using DataTile's automated delivery system. This ensures the accurate and seamless delivery of new waves, reports, and dashboards without disrupting user tasks. Pre-release staging and quality checks prevent errors and eliminate the need for manual versioning of deliverables.
Research delivery
Research suppliers might lack access to high-stakes clients' infrastructure. By leveraging this feature, they can securely produce research deliverables like databases, dashboards, and reports and effortlessly transfer them to the client's DataTile server.
Exporting an Entity to a dump file
Security requirements
Role:
Database creatorfor exporting databases.Role:
Dashboard creatorfor exporting dashboards.Role:
Syndicate manageraccess to the API for downloading entities as dump files.Entity ownership - to maintain data privacy, only the entity's owner is permitted to export it and provide it for uploading on another server.
Get the entity’s HASH from its URL. Here’s how to acquire it.
Being authenticated on the source server, enter the following URL in the browser:
To download an dump file of the entity ad hoc:
https://<SOURCE_SERVER_URL>/api/v2/entities/<HASH>/remote/export
or
https://<SOURCE_SERVER_URL>/api/v2/entities/<HASH>/remote/export?synced=True
to download the dump file containing credentials to keep the target entity you will create on the new server synchronized with its source.
Once DataTile generates the dump file for you, it will be downloaded promptly.
Afterward, you can create new entities from this dump file by yourself or send it to other users, allowing them to do so on another server.
Keeping dump files secure is crucial, as anyone can use them to restore your entities and access sensitive information. The best practice is to delete dump files once the transfer is complete and they are no longer needed.
Restoring an Entity from Dump Files on a New Server
Security requirements
Role:
Database creatorfor creating databases.Role:
Dashboard creatorfor creating dashboards.Role:
Syndicate managerallows user to restore entities from a dump file.
To create a new entity from a dump file, authenticate on the target server and do the following:
Click on the '+' icon in the Drive
‘Entity from dump file’
Select the dump file from your disk
Click Upload.
The newly created entity will be owned by the account under which it was restored. You can manage it as usual by sharing, granting access, or deleting it.
Be mindful of dependencies between the entities you plan to restore. For instance, dashboards are built on top of databases. Transfer and restore entities one at a time, taking into account their internal dependencies. For example, before restoring a dashboard, ensure the required databases already exist in the target folder.
The source and the target servers are supposed to have the same versions. DataTile may refuse the dump file created by the server of another version. I.e. exported packages become stale if you keep them for too long.
Triggering cross-server synchronization of entities
Requirements
Role: Syndicate manager access to the API for downloading entities as dump files.
Entity ownership - to maintain data privacy, only the entity's owner is permitted to export it and provide it for uploading on another server.
When you create a dump of an entity with synchronization credentials, its icon changes to signify that this entity now serves as a source for remote entities.
You can get information about remote entities, the time of last synchronization, etc. in the info panel of an entity.
Synchronization is not fully automatic. You can edit source entities, reload data, and test them before explicitly directing DataTile to propagate the changes.
When you're ready to distribute the updated entities, manually invoke the 'publish' API call by entering the following URL in the browser while authenticated on the source server:
https://<SOURCE_SERVER_URL>/api/v2/entities/<HASH>/remote/publish
where HASH identifies the entity.
As for creation, try to publish synchronized entities in the order of their dependency to minimize potential dysfunctions of the dependent entities.
Please be aware of the following limitations for synchronized entities:
Reduced editing functionality: The editing capabilities of synchronized entities are limited to maintain consistency with their source.
Cloning prohibition: Synchronized entities cannot be cloned.
Dumping prohibition: Creating dump files from synchronized entities is not allowed.