It’s been quite a while since my last blog post. However, this topic has the potential to be so transformational, I just had to get my thoughts down.
Why read on? Unlock next-level styling capabilities for your portal project 😎
Be sure to check out part 2 for how to retrieve File columns using the Portal Web API
Previously, I used the Notes table as a workaround for all my portal multimedia needs; to upload and retrieve cover artwork and sound files for tracks in my Spotify clone project, attaching due diligence and evidence documentation in my case management portals, etc.
Relatively recently, Dataverse introduced the File and Image column types. The catch was that these weren’t first class citizens of the portal and their contents couldn’t be accessed with either FetchXML or the Portal Web API. With the recent release of Read operations in the Portal Web API, all this changes 😎
Benefits of these new column types include:
- The ability to apply field-level security in the portal web API and limit access to only specific image columns
- Structured data: ease of management in the back-end e.g. programmatically check / check at a glance which documents have been received – no need to trawl through endless notes with meaningless filenames
- Data storage benefits: out-of-the-box integration with Azure blob storage:
“Thumbnail images and image metadata are stored in Microsoft Dataverse, which includes the information needed to retrieve the full image. Full images are stored in file storage on the Azure blob to reduce data storage consumption.” (Source: Image columns (Microsoft Dataverse) – Power Apps | Microsoft Docs).
Unless you’re using Customer Managed Keys, files uploaded to the File column type are also stored in Blob storage rather than Dataverse relational storage
How to retrieve full size images using the Portal Web API
The short version – only the answer
Like with Create, Update and Delete operations, we first need to get an authentication token before using the Portal Web API. For ease of re-use, I save this to a separate Web Template and include when required (see here for how to do so). Note – this needs to be included in every page where you make use of the API… it’s not just a matter of once per user session
Next, we need to request the image in the webapi.safeAjax function
The request type is GET (obvious I guess, as we are getting data as opposed to posting it)
The URL includes 4 key pieces of information:
- The database / logical name of the table that contains the image column
- The GUID of the record we require (I’m getting this dynamically from the id URL query string parameter e.g. https://eventmanagement2.powerappsportals.com/event-read-portal-web-api/?id=f7007433-a743-4054-9058-1ec4bdae10c4)
- The database / logical name of the image field
- What to retrieve: we can either retrieve the binary contents of the image or an object that includes the base64 contents of the image (as well as some metadata) – we’ll use base64 as it’s easy to work with in HTML
Note – by default, the API will only retrieve a thumbnail-sized image – a 144px x 144px resolution square version taken from the original (and cropped if necessary) rather than the full sized image. We’ll use the size=full parameter to get around this
Retrieve a full-sized image for a single record
webapi.safeAjax({ type: "GET", url: "/_api/musdyn_tracks({{request.params.id}})/musdyn_artwork?size=full", contentType:"application/json", success: function(res) { $('body').append('<img class="img-responsive" src="data:image/png;base64,' + res.value + '">'); }, error: function(xhr, status, error){ $('body').append('<img class="img-responsive" src="/artwork-not-found.svg">'); var errorMessage = xhr.status + ': ' + xhr.statusText; console.log(errorMessage); } });
In this example, I’ve uploaded a fallback image to Web Files in the Portal Management app. Always good to provide graceful degradation.
The longer painful version – my journey and lessons learnt arriving at the answer
Confession – I don’t know everything!! I really struggled trying to figure this out for myself. At first I could only download the thumbnail. Then I tried adding $value?size=full at the end of the request URL to get the full sized image. Here’s what the result looked like – no idea how to work with this byte array (WTF is a byte array?!!). Besides looking a mess, it was very slow to retrieve:
Tip – If you can’t find help specific to the Portal Web API, it’s well worth looking to the Dataverse Web API. That’s what I did. Nishant Rana’s extremely helpful blog post on Using File and Image Data Type in Dataverse (Dynamics 365)
Nishant very helpfully points out that the image can be requested as either a byte array or in base64 format. We specify this by adding parameters to the end of the request URL like so:
Byte array:
/_api/msevtmgt_events(f7007433-a743-4054-9058-1ec4bdae10c4)/px3_heroimage/$value?size=full
Base64:
/_api/msevtmgt_events(f7007433-a743-4054-9058-1ec4bdae10c4)/px3_heroimage?size=full
Base64 is what we want. However, what the API returns is an object. The actual contents of the image are in an attribute called value:
Therefore to output the image, we use res.value:
$(‘body’).append(‘<img class=”img-responsive” src=”data:image/png;base64,’ + res.value + ‘”>’);
Retrieve thumbnail images from multiple records
In my Spotify clone project, there were many cases where I needed to retrieve images from multiple records within a single web template e.g. when viewing a playlist, a list of liked tracks or a list of albums for a given artist. To achieve this, we’d amend our URL slightly. Instead of specifying a GUID, we’ll use the top parameter to specify the number records we wish to retrieve images from…
webapi.safeAjax({ type: "GET", url: "/_api/msevtmgt_events?$select=px3_heroimage&$filter=px3_heroimage_url ne null&$orderby=px3_heroimage_timestamp desc", contentType:"application/json", success: function(res) { $( res.value ).each(function( index ) { $('#main').append('<img class="img-responsive" src="data:image/png;base64,' + res.value[index].px3_heroimage + '">'); }); }, error: function(xhr, status, error){ $('body').append('<img class="img-responsive" src="/artwork-not-found.svg">'); var errorMessage = xhr.status + ': ' + xhr.statusText; console.log(errorMessage); } });
Note – I failed to figure out the URL format for retrieving multiple full size images 😢 but I did come up with a workaround…
- Retrieve the GUIDs of the top n most recent records that contain images
- Loop through each record
- Retrieve the full size image for the record with that GUID
- Output the image
webapi.safeAjax({ type: "GET", url: "/_api/msevtmgt_events?$select=msevtmgt_eventid&$filter=px3_heroimage_url ne null&$orderby=px3_heroimage_timestamp desc", contentType:"application/json", success: function(res) { $( res.value ).each(function( index ) { webapi.safeAjax({ type: "GET", url: "/_api/msevtmgt_events(" + res.value[index].msevtmgt_eventid +")/px3_heroimage?size=full", contentType:"application/json", success: function(res) { $('#main').append('<img class="img-responsive" src="data:image/png;base64,' + res.value + '">'); }, error: function(xhr, status, error){ $('body').append('<img class="img-responsive" src="/artwork-not-found.svg">'); var errorMessage = xhr.status + ': ' + xhr.statusText; console.log(errorMessage); } }); }); }, error: function(xhr, status, error){ var errorMessage = xhr.status + ': ' + xhr.statusText; console.log(errorMessage); } });
So what are these id, timestamp and url columns?
Each image column has the following supporting fields; id, timestamp and url
The database / logical name for my image column is musdyn_artwork
The supporting columns for this are:
musdyn_artworkid | The unique identifier for the image. This can be used to check whether an image has been attached before you waste time / bandwidth attempting to retrieve image data.
Example: 0638d5e5-1297-ec11-b400-000d3aca27a4 |
musdyn_artwork_timestamp | When the image was last updated (useful for caching purposes as well as sorting).
Example: 637814835979366483 |
musdyn_artwork_url | A download link for the image (Note – this is a Dynamics / Model-Driven App URL rather than portal URL so is no use to external users).
Example: https://yourorg.crm11.dynamics.com/Image/download.aspx?Entity=msevtmgt_event&Attribute=px3_heroimage&Id=f7007433-a743-4054-9058-1ec4bdae10c4&Timestamp=637814835979366483 |
In the following example, I first use the entities object to retrieve the record then check for the presence of an image using liquid. If my liquid condition returns true, only then do I retrieve the image:
{% assign track_record = entities['musdyn_track'][request.params.id] %} {% if track_record.musdyn_artworkid %} <script> webapi.safeAjax({ type: "GET", url: "/_api/musdyn_tracks({{ request.params.id }})/musdyn_artwork/?size=full", contentType:"application/json", success: function(res) { $('#main').append('<img class="img-responsive" src="data:image/png;base64,' + res.value + '">'); }, error: function(xhr, status, error){ $('#main').append('<img class="img-responsive" src="/artwork-not-found.svg">'); var errorMessage = xhr.status + ': ' + xhr.statusText; console.log(errorMessage); } }); </script> {% endif %}
Further considerations
Dataverse doesn’t optimise images for the web so it’s on us to do so – whether that be manually in something like Photoshop or Gimp, or automated using Power Automate and Encodian’s excellent connector.