How to decode metadata_storage_path produced by Az

Question

Using .NetCore 1.1.2.

After successfully getting results from a search via Azure Search SDK, I am trying to decode the metadata_storage_path value. I've seen people saying to use HttpServerUtility.UrlTokenDecode in .NET or an equivalent in other languages as seen here.

Then the question becomes, what is the equivalent in .NetCore of HttpServerUtility.UrlTokenDecode? With:

var pathEncoded = "aHR0cHM6Ly9mYWtlZC5ibG9iLmNvcmUud2luZG93cy5uZXQvcGRmYmxvYnMvYW5udWFsX3JlcG9ydF8yMDA5XzI0NTU20";

I have tried the following:

var pathbytes = Convert.FromBase64String(pathEncoded); 
//Throws System.FormatException "Invalid length for a Base-64 char array or string."

and

var pathbytes = WebEncoders.Base64UrlDecode(pathEncoded);
//Throws System.FormatException - "TODO: Malformed input."

Interestingly enough, everything works just fine if I cut off the last charater in pathEncoded... What is the proper way to handle this situation with Microsoft.AspNetCore 1.1.2?

Answer 1

HttpServerUtility.UrlTokenEncode appends an extra trailing character to the encoded string. You're doing it right - just remove that extra character and use WebEncoders.Base64UrlDecode. See this Q&A for details.

Answer 2

I used the following function in asp.net core 2.1 to encode the meta_storage_path value from Azure search.

private string DecodeBase64String(string encodedString)
{
    var encodedStringWithoutTrailingCharacter = encodedString.Substring(0, encodedString.Length - 1);
    var encodedBytes = Microsoft.AspNetCore.WebUtilities.WebEncoders.Base64UrlDecode(encodedStringWithoutTrailingCharacter);
    return HttpUtility.UrlDecode(encodedBytes, Encoding.UTF8);
}

Answer 3

I just wanted to add that you can also deselect the 'Base-64 Encode Keys' Azure search indexer option.

NOTE: Only do this for fields with no characters Azure considers invalid for document keys.