How to automate download of weekly export service

问题:

In SalesForce you can schedule up to weekly "backups"/dumps of your data here: Setup > Administration Setup > Data Management > Data Export

If you have a large Salesforce database there can be a significant number of files to be downloading by hand.

Does anyone have a best practice, tool, batch file, or trick to automate this process or make it a little less manual?

回答1:

Last time I checked, there was no way to access the backup file status (or actual files) over the API. I suspect they have made this process difficult to automate by design.

I use the Salesforce scheduler to prepare the files on a weekly basis, then I have a scheduled task that runs on a local server which downloads the files. Assuming you have the ability to automate/script some web requests, here are some steps you can use to download the files:

1. Get an active salesforce session ID/token
   • enterprise API - login() SOAP method
2. Get your organization ID ("org ID")
   • Setup > Company Profile > Company Information OR
   • use the enterprise API getUserInfo() SOAP call to retrieve your org ID
3. Send an HTTP GET request to https://{your sf.com instance}.salesforce.com/ui/setup/export/DataExportPage/d?setupid=DataManagementExport
   • Set the request cookie as follows:
     • oid={your org ID}; sid={your session ID};
4. Parse the resulting HTML for instances of <a href="/servlet/servlet.OrgExport?fileName=
   • (The filename begins after fileName=)
5. Plug the file names into this URL to download (and save):
   • https://{your sf.com instance}.salesforce.com/servlet/servlet.OrgExport?fileName={filename}
   • Use the same cookie as in step 3 when downloading the files

This is by no means a best practice, but it gets the job done. It should go without saying that if they change the layout of the page in question, this probably won't work any more. Hope this helps.

回答2:

A script to download the SalesForce backup files is available at https://github.com/carojkov/salesforce-export-downloader/

It's written in Ruby and can be run on any platform. Supplied configuration file provides fields for your username, password and download location.

With little configuration you can get your downloads going. The script sends email notifications on completion or failure.

It's simple enough to figure out the sequence of steps needed to write your own program if Ruby solution does not work for you.

回答3:

I'm Naomi, CMO and co-founder of cloudHQ, so I feel like this is a question I should probably answer. :-)

cloudHQ is a SaaS service that syncs your cloud. In your case, you'd never need to upload your reports as a data export from Salesforce, but you'll just always have them backed up in a folder labeled "Salesforce Reports" in whichever service you synchronized Salesforce with like: Dropbox, Google Drive, Box, Egnyte, Sharepoint, etc.

The service is not free, but there's a free 15 day trial. To date, there's no other service that actually syncs your Salesforce reports with other cloud storage companies in real-time.

Here's where you can try it out: https://cloudhq.net/salesforce

I hope this helps you!

Cheers, Naomi

回答4:

Be careful that you know what you're getting in the back-up file. The backup is a zip of 65 different CSV files. It's raw data, outside of the Salesforce UI cannot be used very easily.

回答5:

Our company makes the free DataExportConsole command line tool to fully automate the process. You do the following:

1. Automate the weekly Data Export with the Salesforce scheduler
2. Use the Windows Task Scheduler to run the FuseIT.SFDC.DataExportConsole.exe file with the right parameters.

回答6:

I recently wrote a small PHP utility that uses the Bulk API to download a copy of sObjects you define via a json config file.

It's pretty basic but can easily be expanded to suit your needs.

Force.com Replicator on github.

回答7:

Adding a Python3.6 solution. Should work (I haven't tested it though). Make sure the packages (requests, BeautifulSoup and simple_salesforce) are installed.

import os
import zipfile
import requests
import subprocess
from datetime import datetime
from bs4 import BeautifulSoup as BS
from simple_salesforce import Salesforce

def login_to_salesforce():
    sf = Salesforce(
        username=os.environ.get('SALESFORCE_USERNAME'),
        password=os.environ.get('SALESFORCE_PASSWORD'),
        security_token=os.environ.get('SALESFORCE_SECURITY_TOKEN')
    )
    return sf

org_id = "SALESFORCE_ORG_ID"  # canbe found in salesforce-> company profile
export_page_url = "https://XXXX.my.salesforce.com/ui/setup/export/DataExportPage/d?setupid=DataManagementExport"

sf = login_to_salesforce()
cookie = {'oid': org_id, 'sid':sf.session_id}
export_page = requests.get(export_page_url, cookies=cookie)
export_page = export_page.content.decode()

links = []
parsed_page = BS(export_page)
_path_to_exports = "/servlet/servlet.OrgExport?fileName="
for link in parsed_page.findAll('a'):
   href = link.get('href')
   if href is not None:
       if href.startswith(_path_to_exports):
           links.append(href)

print(links)
if len(links) == 0:
    print("No export files found")
    exit(0)

today = datetime.today().strftime("%Y_%m_%d")
download_location = os.path.join(".", "tmp", today)
os.makedirs(download_location, exist_ok=True)
baseurl = "https://zageno.my.salesforce.com"

for link in links:
    filename = baseurl + link
    downloadfile = requests.get(filename, cookies=cookie, stream=True)  # make stream=True if RAM consumption is high
    with open(os.path.join(download_location, downloadfile.headers['Content-Disposition'].split("filename=")[1]), 'wb') as f:
        for chunk in downloadfile.iter_content(chunk_size=100*1024*1024):  # 50Mbs ??
            if chunk:
                f.write(chunk)