Step 3: How to Upload Data
Synapse.org is the data management software for all data hosted and shared on the ELITE Portal. Think of Synapse as the back-end and the ELITE Portal as the front-end for your data.
The EL DMCC has configured certain Projects in Synapse to populate the tables you see under the ‘Explore’ heading of the ELITE Portal. These Projects have a specific folder structure and access permissions applied to them to allow you (as the data contributor) and the EL DMCC to collaboratively curate and annotate the data. When you upload data to these designated Synapse Projects, the files will be connected to the ELITE Portal and visible only to users who have view permission. Data files become visible to the public after Step 5: How to Release Data is complete.
You will be uploading data into Synapse using a variety of tools. This article will give you a brief guide to ensure you upload data to the correct place, using the appropriate methods.
Uploading data involves the following actions, to be completed in order:
1. Ensure your account is certified
If your Synapse account is certified, there will be an icon indicating so on your profile page. If it is not yet certified, see Step 2: How to Become Synapse Certified for instructions.
2. Locate your Synapse project
In order to upload data, you need to have a designated Synapse project, which will be provided to you once you open an AD + EL Service Desk ticket with the EL DMCC. If you haven’t started an AD + EL Service Desk ticket yet, see Step 1.1: How to Submit a Service Desk Request for more information and instructions on how to do this.
You should have been provided a Synapse project space to house your data. Here’s how to find this:
Log into Synapse.org
Click on your profile picture in the upper right hand corner and click Projects
Click on your project—the project title should match the information you provided the EL DMCC data managers in your Service Desk ticket
3. Locate the folder(s) for your data
Once you’re in your project, click on Files. The EL DMCC will have set up staging folders titled with the study name and data types you indicated in your intake form. These staging folders are private and accessible only to your team and the EL DMCC team.
Example:
staging_human_serum_determinants_of_aging_proteomics
staging_human_serum_determinants_of_aging_metabolomics
You can find more information on Synapse projects, files, and folders on the Synapse docs site.
4. Upload your data
Now that you have an appropriate folder to house your data, you’re ready to upload. You can do this using the Synapse.org UI or by using one of our programmatic clients—Python, R, or command-line. For small folders/files, the website upload is generally easiest. For larger uploads (large files or many files), we recommend using the programmatic clients.
Especially for larger uploads, we recommend authenticating your account with a personal access token, rather than a password, to prevent timeout during a data upload.
To upload using the web UI:
navigate to the folder you want to upload to and click Folder Tools → Upload or Link to a File.
To upload using the Python client:
To upload using the command-line client (requires the Python client):
To upload using the R client:
To upload files that already exist in an S3 bucket or Google Cloud Storage Bucket:
As an alternative option, if your files are already on GEO, SRA, or another platform that doesn’t require a login, you can link directly to the download HTTPS or FTP URL (for example, the links under the Data Access tab on this SRA page). Here’s how:
in your Synapse account, under Folder Tools, click Upload or Link to a File
simply add in the link
Using this method, users will be able to download the file as if it lived on the ELITE Portal/Synapse, but you only need to upload it once! Please note that annotation on Synapse is still required with this method.
If you have any questions about whether your repository of choice is compatible with this, please reach out to us through your AD + EL Service Desk ticket.
6. Annotate your data
Once your dataset is uploaded, you must annotate it in order to make it findable and understandable. You can find those instructions at Step 4: How to Annotate Data