Large Data Volumes in Salesforce

Introduction

In Salesforce it's possible to process large amounts of records. We can update, delete, and manually or automatically insert records as needed.

What is the best tool for processing large amounts of data? While there is no perfect tool, our team at Tero by Nearsure can help you optimize Salesforce data management tools according to your project’s needs and characteristics.

Data Import Wizard

Data Import Wizard is one of the most basic tools that we recommend. You can use this interface anytime without downloading an application or performing a previous configuration. Data Import Wizard is integrated into your environment.

Objects allowed:

Leads
Accounts
Contacts
Custom Objects
Solutions

Record Limit

Limited to 50,000 records per import.

Steps

Login into Salesforce.
From Setup, enter “Data Import Wizard” in the Quick Find box, then select Data Import Wizard.
Click on the Launch Wizard button.
Select the type of data that you want to import. Click on Standard Objects or click on Custom Objects to import custom objects.
Select the operation you need to perform.
Specify matching and other criteria as necessary.
Specify the file that contains your data. (CSV)
Map the fields in your CSV with the fields in your Salesforce instance related to the object type. (Your import file can be up to 100 MB.)
Finally you are redirected to a detail page where you can see the import status of the import.

Operations on selected objects

Add new records
Updating existing records
Add new and update existing records

Advantage:

This tool is intuitive and user-fiendly. You can choose whether or not to trigger workflows with the import.

Disadvantages:

Can only import data of Accounts, Contacts, Leads, Solutions and Custom objects.

Workbench

Workbench is a powerful visual tool that allows us to manage Salesforce data. It provides numerous useful functionalities. To access this tool we don't need to download an application either, just simply log-in to a website.

Objects allowed

Standardand custom objects.

Record Limit

More than 5 million records.

Steps

Log in to your organization.
Open a new browser tab and navigate to https://workbench.developerforce.com/login.php
Log in to Workbench and allow access to your organization.

Operations on selected objects

Insert, Update, Upsert, Delete, and Export. Workbench also supports the Undelete Program,Deploy, Retrieve, Rest Explorer, and Apex Execute actions too.

Advantages

In the browser, it is easy toread a table due to simplified result information.

Disadvantages

Workbench cannot save and reuse data load assignments, as we can do with the data loader.

Salesforce Data Loader

A Salesforce Data Loader is a universal tool used for bulk importing and exporting of data. A Data Loader can input, update, and completely delete large amounts of Salesforce records. It provides a user-friendly UI or command line access when importing records, and will read, extract, and load information from CSV files or a database. For this tool it is necessary to download an application.

Objects allowed

Support for all objects, including custom objects.

Record Limit

Up to 5 million records.

Steps

Log in to your Salesforce application.
From Setup, enter “Data Management” in the Quick Find box, then select Data Loader.
Install that downloaded file in your machine.
To start Data Loader double click on the shortcut on your desktop.

Operations on selected objects

Insert
Update
Upsert
Delete
Export

Advantages

Drag-and-drop field mapping
Detailed success and error log files in CSV format
Batch size can be specify

Disadvantages

Needs to be downloaded

Batch Apex

Batch Apex is commonly used to process larger operations that would normally be restricted by governor limits. Using Batch Apex, you can process records asynchronously in batches. This option is most often used by Salesforce developers because it requires writing code. Here we show you how to create a batch class.

Batch Apex Syntax

To write a Batch Apex class, your class must implement the “Database.Batchable” interface and include the following three methods:

Start
Execute
Finish

global class MyBatchClassexample implements Database.Batchable<sObject> {

    global (Database.QueryLocator | Iterable<sObject>) start(Database.BatchableContext bc) {
        //collect records for processingc
    }
    global void execute(Database.BatchableContext bc, List<P> records){
        // process each batch of records
    }    
    global void finish(Database.BatchableContext bc){
        // execute any post-processing operations
    }    
}

Apex

Invoking a Batch Class

To invoke a batch class, you must instantiate it and then invoke the method “Database.executeBatch”, together with the instance you’ve previously created:

MyBatchClassexample myBatch = new MyBatchClassexample(); 
Id batchId = Database.executeBatch(myBatch);

Apex

You can also choose to pass a second scope parameter to specify the number of records that should be passed into the execute method for each batch.

Id batchId = Database.executeBatch(myBatch, 100);

Apex

Monitor Bulk Data Load Jobs

To track the status of Bulk Upload jobs that are either in progress or recently completed, enter “Bulk Upload Jobs” in the Quick Find box, then select Bulk Upload Jobs. This page allows you to monitor the progress of current jobs and the results of recent jobs.

‍

References: