Glen Mazza's Weblog

Salesforce Trailhead's Create a Custom Channel and Enrich Change Events tutorial shows how to create new Change Data Capture (CDC) channels providing "enriched" messages, i.e., those having certain fields always present. While CREATE and UNDELETE event messages always provide all non-null fields, UPDATE and DELETE events do not, but event enriching can be used for the latter two cases. For example, in my previous tutorial, it would be more readable to always return the Account Name in update and delete events so it can be logged alongside the Account Salesforce ID. And the tutorial gives an example of creating a custom External Account ID field in Salesforce that can be provided in every message and used to identify an account record in the external system for synchronization.

Some drawbacks/limitations of event enriching:

• It is no longer possible to send multiple changes in one event message (e.g., receiving one message that 100 Salesforce IDs of a given entity had the same field changed to a specific value), as each record will have a separate enriched field value.
• The enriched fields cannot contain elements from other entities beyond their Salesforce ID, so for a Contact message one can get the Contact's Account ID but not properties of the Account such as Name.
• It is more cumbersome to create and modify custom channels having enriched fields than relying on the default behavior from the standard CDC streams. The Salesforce Setup UI lacks this functionality, one instead can work with the Salesforce Postman project (GitHub) making REST calls against either Tooling or Metadata APIs. The Salesforce tutorial effectively shows the process using the former.

I expanded my CDC listener sample from the previous tutorial to allow for reading from a custom channel "CDCSample__chn" instead of the standard AccountChangeEvent. The channels can be switched just by adjusting the configuration file and restarting the application. Unfortunately, I was unable to get the Account "Name" property to always be provided in the message as an enriched field. I tried five or six other Account fields and they all worked fine, I'm not sure what the problem with Name is, whether my error or a Salesforce bug.

Using Salesforce's Postman project and the Trailhead tutorial, below are the commands I ran to create and configure this custom channel. Main thing to keep in mind is that a channel is a holder for message event streams, but not a message stream itself. A channel contains one or more channel members, with each member providing messages on a particular entity. It is also with the channel member that any desired enriched fields are defined.

• Creating the custom Platform Event Channel:

  POST to: {{_endpoint}}/services/data/v{{version}}/tooling/sobjects/PlatformEventChannel
  
  with body:
  {  
    "FullName": "CDCSample__chn",
    "Metadata": {
      "channelType": "data",
      "label": "Custom Channel for Change Data Capture Sample"
    }
  }
  

  There should be a success message giving the Salesforce ID of the new channel. Metadata on the PlatformEventChannel can be queried using the following GET call:

  {{_endpoint}}/services/data/v{{version}}/tooling/query/?q=SELECT Id, FullName FROM PlatformEventChannel WHERE DeveloperName='CDCSample'
  

• Creating a PlatformEventChannelMember in the PlatformEventChannel. Here I'm choosing three fields to always appear.

  POST to {{_endpoint}}/services/data/v{{version}}/tooling/sobjects/PlatformEventChannelMember
  
  {
    "FullName": "CDCSample_chn_AccountChangeEvent",
    "Metadata": {
      "enrichedFields": [
        {
          "name": "Industry"
        },
        {
          "name": "Name"
        },
        {
          "name": "TickerSymbol"
        }
      ],
      "eventChannel": "CDCSample__chn",
      "selectedEntity": "AccountChangeEvent"
    }
  }
  

  The success message provides the ID for the Channel Member, whose details can be later queried with a GET similar to the following:

  {{_endpoint}}/services/data/v{{version}}/tooling/sobjects/PlatformEventChannelMember/0v85e0000004Cl7AAE
  

  Sample output:

  {
      "attributes": {
          "type": "PlatformEventChannelMember",
          "url": "/services/data/v51.0/tooling/sobjects/PlatformEventChannelMember/0v85e0000004Cl7AAE"
      },
      "Id": "0v85e0000004Cl7AAE",
      "IsDeleted": false,
      "DeveloperName": "CDCSample_chn_AccountChangeEvent",
      "Language": "en_US",
      "MasterLabel": "AccountChangeEvent",
      "NamespacePrefix": null,
      "ManageableState": "unmanaged",
      "CreatedDate": "2021-05-15T16:01:21.000+0000",
      "CreatedById": "0055e000000nMxcAAE",
      "LastModifiedDate": "2021-05-19T10:27:59.000+0000",
      "LastModifiedById": "0055e000000nMxcAAE",
      "SystemModstamp": "2021-05-19T10:27:59.000+0000",
      "FullName": "CDCSample_chn_AccountChangeEvent",
      "Metadata": {
          "enrichedFields": [
              {
                  "name": "Industry"
              },
              {
                  "name": "Name"
              },
              {
                  "name": "TickerSymbol"
              }
          ],
          "eventChannel": "CDCSample__chn",
          "selectedEntity": "AccountChangeEvent",
          "urls": null
      },
      "EventChannel": "0YL5e0000008OIAGA2",
      "SelectedEntity": "AccountChangeEvent"
  }
  

  Above query provides the enriched fields, but for links to them specifically, this query can be used:

  {{_endpoint}}/services/data/v{{version}}/tooling/query/?q=SELECT Id,ChannelMemberId,Field FROM EnrichedField ORDER BY ChannelMemberId
  

• Keep this tutorial page handy for whenever it is needed to adjust the enriched fields of a platform event channel member. It involves PATCH requests to the endpoint originally used to create the stream.

I posted to Github a Spring Boot-based client library for making OAuth2-enabled REST calls to Salesforce's API. Supported are Salesforce's JWT Bearer Token and username/password flows discussed in my earlier blog post. The library supports use of Salesforce's REST API, SOQL Query, and Apex REST functionality. It uses Spring Security's OAuth 2 client to obtain access tokens necessary for making these calls.

The integrated test cases give examples of the client in action. As they involve creating, updating, and deleting Salesforce Accounts they should be run against a non-production instance. Salesforce offers free developer instances you can sign up for. Note the test case for the Apex REST functionality will require installing this Apex REST endpoint from the Salesforce documentation. To run the tests, first create an application-test.properties file in the itest resources folder with the configuration necessary for the flow you are using. There is a template file in that folder specifying what is needed for each OAuth2 flow type. For usage of this library by other applications, this configuration would be placed in the importing application's properties file. The library's SalesforceOAuth2Config class reads that configuration, and will halt on startup with informational messages if anything needed is missing. Once done, the integrated tests can be run from IntelliJ or command-line via ./gradlew integratedTest.

The Username/Password flow is supported out of the box by Spring, but the JWT bearer token flow requires some extra classes that I implemented following Spring's source code for their standard password and client grant flows:

• SalesforceJwtBearerOAuth2AuthorizedClientProvider - For each outgoing request to a resource server, checks if the JWT Bearer Token method is being used for access tokens and also if an authorized client (i.e., one with a valid access token) is already available. If yes and no, respectively, initiates the process to obtain a new access token to use with the resource API call.
• OAuth2SalesforceJwtBearerGrantRequest - created by above class, it holds the information necessary to create the JWT bearer token for making an access token request.
• OAuth2SalesforceJwtBearerGrantRequestEntityConverter - Constructs the JWT bearer token
• SalesforceJwtBearerTokenResponseClient - Uses the JWT Bearer token created by the above class to obtain a new access token from the authorization server.

Within the app, SOQL queries are handled by the SOQLQueryRunner which provides two options for responses: a JSON-formatted string or a developer-defined parameterized implementation of SOQLQueryResponse (example in the integrated test). The latter takes advantage of the fact that SOQL queries share much common structure and need only a relatively small portion to be overridden to hold fields specific to the SOQL query.

What happens when access tokens expire? The WebClient calls have a retry(1) setting that allows for one additional call to the resource server in case of an error such as using an expired access token. In such cases, for the first call, the failure handler in SalesforceOAuth2Config removes the authorized client instance (which has the invalid access token) from memory. For the retry call, SalesforceJwtBearerOAuth2AuthorizedClientProvider notes that there is not an authorized client instance anymore so proceeds to obtain a new access token to allow the second call to proceed. This functionality can be verified by revoking the access token from either Salesforce Setup's Session Management screen or from Connected Apps usage, and confirming that a subsequent resource API call still provides the data. Code breakpoints can also be used to confirm another access token was requested.

Additional Resources

• OAuth 2.0 Client - official Spring documentation
• Carey Development - Blog contains many useful tutorials, including on logging exceptions and reporting them to clients helpful for the creation of this client library.
• Spring WebClient – GET, PUT, POST, DELETE examples
• Spring 5 WebClient and WebTestClient Tutorial with Examples
• Salesforce Username/Password Flow Example - Very helpful in creating the Username-Password flow option in this library.

Salesforce's Mohith Shrivastava's second Connected App tutorial shows how the JWT Bearer Token Flow can be used with Postman to obtain access tokens. Separately, SFDCStop shows how to use an access token to retrieve data from Salesforce. Summary of the process for getting an access token via JWT Bearer Token flow is given below, along with the alternative Username-Password flow.

Obtaining Access Token via JWT Bearer Token Flow

Mohith's second tutorial largely follows the Trailblazer documentation for creating the JWT necessary for obtaining an access token. First, within Postman, open a POST request to https://test.salesforce.com/services/oauth2/token (or login.salesforce.com for production) with two query attributes, grant_type hardcoded to urn:ietf:params:oauth:grant-type:jwt-bearer and assertion with a value that can be generated at JWT.io as follows:

1. Place {"alg":"RS256"} in the JWT header.

2. Place the following in the JWT payload, updating the iss value with the Client ID of the connected app, the aud value (should be either https://test.salesforce.com for sandbox instances, https://login.salesforce.com for all others, including developer instances), the sub with the Salesforce resource owner (the user ID, usually expressed as an email), and exp value being the ten-digit current UNIX timestamp. Ensure there are no spaces or carriage returns in the payload:

   {"iss":"--client id--","sub":"--salesforce user--","aud":"https://test.salesforce.com","exp":"1616865336"}

   You may also wish to add a "jti" field (JWT ID), the value of which can be a randomly generated UUID. As stated in the Trailblazer documentation linked above, if a "jti" field is added, Salesforce will make sure the JWT hasn't been used to make a prior access token request, serving as a guard against replay attacks. If it has been used, it will return a 400 response code with message { "error": "invalid_request", "error_description": "jti already in use"}.

3. Finally, in the Verify Signature portion, as generated in the previous tutorial, place the PublicCert.crt contents and the private key found in key.pem (including the BEGIN/END delimiters) in the corresponding boxes. JWT.io should report "Signature Verified", upon which you can copy the left-side as the assertion value into the Postman call. Make the call and you should get an access token in response.

   

Obtaining Access Token via Username-Password Flow

This process is much more straightforward as no keys are used. As shown in the Salesforce documentation, you can simply make a POST request with the following five query parameters: grant_type, client_id, client_secret, username, and password.

Making Salesforce calls using the Access Token

As demo'ed in the SFDCStop video, can create a GET request to say https://yoursalesforceinstance.salesforce.com/services/data/v50.0/query?q=SELECT+Name,Type+FROM+Account. Under the Authorization tab, select "Bearer", copy-and-paste the access token in, and make the API call.

More information on SOQL calls are available here.

Salesforce Connected Apps provide the conduit for data transfers between Salesforce and third-party apps. Salesforce's Mohith Shrivastava offers four-part series on Connected Apps, roughly four hours of instruction in total, helpful for visual walk-through of configuration. Below are simple steps I've followed for configuring a Connected App, but as always the official documentation should be at least reviewed to make sure all security matters are properly taken care of.

Creating credentials to access a Connected App.

The OAuth Authorization Flows section in the Salesforce Trailblazer documentation details the various methods (or "flows") available for working with Salesforce data externally. Below lists configuration for two of the flows usually best for server-to-server integration, i.e., without an active user logging in and activating the processes. For this use case, Salesforce recommends creating an integration user. Instructions here and here seem to best for creating an integration user. However, the profile options for choosing "API Enabled" and "API Only User" (i.e., user can not directly log into Salesforce) appear not to be under a "Administrative permissions" section anymore but under the System: System Permissions option when one creates the integration user's profile.

• OAuth 2.0 JWT Bearer Flow: Accessing the application using a public/private key pair. Here, the public cert is uploaded to the Salesforce connected app, while the requests are signed by the client using the private key. The Connected App relies on the successful decrypting of the message to confirm the request is coming from the client. In this case, the user's login and connected app consumer key are needed by the client, but not the user's password or the connected app client secret.

  A locally-made key pair is sufficient for this access technique, I've used a combination of the Java keytool and OpenSSL for this. Steps:

  1. Create a Java PKCS12 keystore using the following command. Keystore passwords here don't matter as they will soon be discarded (keys will be worked with directly). Note that validity given below is in days, whatever value you choose, note the key will need to be replaced with it expires:

     keytool -genkeypair -keyalg RSA -sigalg SHA1withRSA -validity 1095 -storepass mypassword -storetype PKCS12 -keystore tempKeystore.p12 -alias myappkey

  2. Create a public certificate to be uploaded to the Salesforce Connected App:

     keytool -export -rfc -keystore tempKeystore.p12 -storepass mypassword -file PublicCert.crt -alias myappkey

  As for the public key itself, it is not needed for making Salesforce calls, but can be obtained if desired via:

  openssl x509 -pubkey -in PublicCert.crt -text -noout

  3. Finally, extracting the private key from the keystore. The private key needs to be kept secure as anyone with it can access the Salesforce Connected App, once the latter is configured with its public cert.

     openssl pkcs12 -in tempKeystore.p12 -passin pass:mypassword -nodes -nocerts -out key.pem
     

• OAuth 2.0 Username-Password Flow for Special Scenarios: Obtaining access tokens using the resource owner's username and password instead of X.509 key pair. Salesforce does not ordinarily recommend this flow, quote: "Use it only if there’s a high degree of trust between the resource owner and the client, the client is a first-party app, Salesforce is hosting the data, and other grant types aren’t available. In these cases, set user permissions to minimize access and protect stored credentials from unauthorized access."

  If you're heavily using Postman to obtain access tokens, this method is much easier and faster than manually creating JWT bearer tokens, of course be careful though that the user and client credentials don't get exposed to others who shouldn't have them.

Salesforce Connected App Configuration

1. Log into Salesforce and click the gear on the upper right side to choose Setup (illustration below). Once the setup window appears, on the left-side search box search and select "App Manager". Once App Manager is up, select the New Connected App button on the right-side.

   

   

2. Click on the "Enable OAuth Settings" checkbox to place in configuration similar to the below. See the official documentation for an explanation of options available, also note the refresh_token role granted below would be unneeded for either of the two flows above (neither use refresh tokens). If using the Bearer Flow, be sure to click on "Use Digital Signatures", place the public key generated earlier into a file and upload. The callback URL will be ignored for either of the two flows, but a value needs to be provided regardless. When done, hit “Save” at the top, and then the “Manage” button, and then “Edit Policies” (covered in next step).

   

3. On the Policies page, here is where you may wish to add IP Address restrictions to limit from where approved clients may call the application. Click Save and scroll to the Profiles section (next step).

   

4. For profiles, add the profile of the User being used to make the Salesforce calls. If an Integration User profile was created as suggested above, for example, choose that. Make sure the user associated with the calls is part of that profile.

5. Back in App Manager, in the table of apps, go to the last column of the new connected app and choose View. As mentioned earlier, just the consumer key will be needed for the Bearer flow, but both that and the consumer secret for the Username-Password flow. As with the private key in the Bearer flow, make sure the client secret is stored confidentially, should it get exposed the connected app should be deleted and a new one which will have a different consumer key and secret created.