Drupal 10 upgrade: File to media

We're continuing our series on upgrading this very website to Drupal 10...but our first proper bit of work to upgrade this site is essentially nothing to do with Drupal 10!

We built this site in the early days of Drupal 8 and Drupal core didn't really handle 'media' any differently to Drupal 7, but we wanted to try out some new modules that included having a library and drag&drop upload. However, these modules are an additional thing that we need to upgrade and have been pain in that sometimes the buttons don't appear, or they're buggy in other ways. Anyway we're going to replace our previous way of doing media handling, with the way that Drupal 8.5 introduced: the media module. To be perfectly honest we could (and maybe should) have done this piece or work a long time ago, but we simply never got around to it. Anyway, here's an idea of what we had before:

On lots of our entity bundles, we had file and image fields, these were named something like field_background_image for example. This would hold the image that was eventually styled into the title header banner at the top of the page. We wanted a nicer experience for uploading and selecting files to use than the core offering at the time. 

We had settled on using the File Entity Browser module which brings with it a lot of dependencies, but then does give a relatively nice experience that looks a bit like this:

You get a drag & drop upload, and you get to pick files from a grid of files if that's what you want.

This is all well and good, but Drupal core basically does all this now, via the media module, so how do we get to this:

To get there, we're going to have to:

• Enable the core media and media library modules
• Set up our media types that we want to use
• Create duplicates of our current file/image fields, but as media fields
• Switch our templating/PHP code over to use the new media fields
• Write some kind of data migration to create new media entities for existing files and update the content to reference the new media entities
• Delete the old fields.

I appreciate that there are modules out there that can help with some of this, but none of them seemed to work quite right for me when I'd tried in the past, and they seemed to get hung up on things like deduplication of files, which I wasn't particularly bothered by.

Setting up core media

This is, by far, the easiest bit: I enable the media module, and then grab the media related config from the core demo umami profile, since that has and image and document media entity all set up and ready to use. I grabbed the following config files:

• media.type.image.yml
• media.type.document.yml
• field.storage.media.field_media_image.yml
• field.storage.media.field_media_document.yml
• field.field.media.image.field_media_image.yml
• field.field.media.document.field_media_document.yml
• core.entity_form_display.media.image.default.yml
• core.entity_form_display.media.document.default.yml

And import them into this site, just as a super quick way to get going with a simple set of config that would enable core media handling.

Duplicating fields

Now for the tedious part: duplicating all those lovely file/image fields with media field equivalents. To get a nice list of the entity bundles that needed this work doing, I went to uninstall the File entity browser module, and Drupal gives me a nice list of the config that would need to be updated. This is essentially a list of bundles where we'd set up our file upload widget that I needed to swap out.

So, I go one by one looking at each bundle, and where there was a field named like: field_background_image for example, I'll create a new entity reference field called: field_background_media and set it to reference media entities. Then I set it to use the media widget, and matched its position and settings on the entity form and entity display modes. I did do this manually, but I imagine this would be fairly simple to automate, but I only had a handful of fields, and I went slowly and carefully with the next bit anyway, making sure to test each of my new fields.

In our theme layer, we do a decent amount with some of the files referenced by these fields, for example the background images for the page headers, get processed by some custom code that generates some specific inline CSS. So I searched the codebase for usages of these field names, and switch them over to the new fields. Being careful that now there was an extra level of entity in the structure of the data, since before I'd have had:

Host entity -> image field -> File entity -> Actual URL to image

But now the new fields will have:

Host entity -> media field -> media entity -> image field -> File entity -> Actual URL to image

This is actually pretty easy for this site, because we'd nicely separated out getting the image from the templates themselves, so this step was fairly straightforward in the end, it was fairly repetitive though. I duplicated a field, found all usages of it, changed them and then tested the code to make sure it was all working properly.

Data migration

Now we have the duplicated fields, and we know they all work we need to get the data into them. We're going to load up any entity that references a file in one of the fields we've duplicated, loop over all the fields, and where there's a file we'll create a new media entity that references this file and insert this into the field on the outer entity. We'll save each entity as we go.

We don't have that much content on this site, so we can run these in an update hook, and the site will be down for a minute or so running the update hook, but that's okay for our use case. Other sites might require a different approach.

Without further ado, here's our code:

/**
 * Migrate image fields to media fields.
 *
 * @param string $entity_type_id
 *   The entity type id.
 * @param array $image_field_map
 *   An array of mappings, each sub array should have the following keys:
 *   - source: The source image field name.
 *   - destination: The destination media field name.
 *   - media_bundle: The media bundle to create.
 * @param string|NULL $bundle_id
 *  The bundle id.
 *
 * @return void
 */
function computerminds_core_migrate_to_media_helper(string $entity_type_id, array $image_field_map, string $bundle_id = NULL): void {
  $entity_storage = \Drupal::entityTypeManager()->getStorage($entity_type_id);

  // Construct a query to find all entities with the image fields.
  $entity_query = $entity_storage->getQuery();
  if (isset($bundle_id)) {
    $entity_query->condition('type', $bundle_id);
  }
  // We don't want to check access.
  $entity_query->accessCheck(FALSE);

  // Add a condition for each image field.
  $field_conditions_group = $entity_query->orConditionGroup();
  foreach ($image_field_map as $field_map) {
    $field_conditions_group->exists($field_map['source']);
  }
  $entity_query->condition($field_conditions_group);

  $entity_ids = $entity_query->execute();

  // Process in chunks of 10.
  foreach (array_chunk($entity_ids, 10) as $entity_ids_chunk) {
    foreach ($entity_storage->loadMultiple($entity_ids_chunk) as $entity) {
      // Process each image field.
      foreach ($image_field_map as $field_map) {
        $image_ids = $entity->get($field_map['source'])->getValue();
        $entity->set($field_map['destination'], NULL);
        foreach ($image_ids as $image_id) {
          $file = \Drupal::entityTypeManager()
            ->getStorage('file')
            ->load($image_id['target_id']);
          if (!$file) {
            $message = 'Media cannot be created. The %entity_type_id with ID: %entity_id of bundle: %bundle refers to the image file with ID: %fid. But there is no information about the file with this ID in the database.';
            \Drupal::logger('image_field_to_media')->error($message, [
              '%fid' => $image_id['target_id'],
              '%entity_type_id' => $entity_type_id,
              '%bundle' => $bundle_id,
              '%entity_id' => $entity->id(),
            ]);
            continue;
          }
          $media_data = [
            'bundle' => $field_map['media_bundle'],
            'uid' => $file->getOwnerId(),
            'created' => $file->getCreatedTime(),
            // @TODO: Make this work for languages.
          ];

          // Asign the file to the correct field.
          switch ($field_map['media_bundle']) {
            case 'image':
              $media_data['field_media_image'] = $image_id;
              break;

            case 'document':
              $media_data['field_media_document'] = $image_id;
              break;
          }
          $media = \Drupal\media\Entity\Media::create($media_data);
          $media->save();
          $entity->get($field_map['destination'])->appendItem($media->id());
        }
      }
      $entity->save();
    }
  }
}

Note that I've hardcoded this with our specific fields on our media entities.

We can then call this helper like this:

function computerminds_core_update_9114() {
  $entity_type_id = 'node';
  $bundle_id = 'article';
  $image_field_map[] = [
    'source' => 'field_banner_image',
    'destination' => 'field_banner_media',
    'media_bundle' => 'image',
  ];
  $image_field_map[] = [
    'source' => 'field_mobile_banner_image',
    'destination' => 'field_mobile_banner_media',
    'media_bundle' => 'image',
  ];
  $image_field_map[] = [
    'source' => 'field_files',
    'destination' => 'field_attachments',
    'media_bundle' => 'document',
  ];

  computerminds_core_migrate_to_media_helper($entity_type_id, $image_field_map, $bundle_id);
}

I then went through and wrote an update hook for each entity type and bundle combination that I needed, following the same boilerplate pattern of an array of fields to map.

Once the update hooks have run, then I was able to check each of the bits of content and see that the media entities had been created correctly, and added correctly added to the entities.

Delete the old fields

The fun, last step! Deleting the old fields: which will remove the field data from the content entities, but not the actual file entities themselves, so they'll still be around to be referenced by the media entities.

This then also allows me to uninstall the File entity browser module, and then the Entity browser, Entity embed module and Dropzonejs module, big win because we now have many fewer modules to upgrade.

Deployment

Our deployments do this:

1. Run database updates
2. Import config
3. Rebuild caches

To deploy this work, I committed each of the above steps, along with their exported config, to git. Then I was able to deploy each commit in turn, so that the site briefly had lots of empty media fields, and wasn't showing any images, but then minutes later had all the content migrated and then finally all the old fields and modules were gone. Oh, I also took a full backup before I started these deployments!

That's it for the first big step on the road to Drupal 10 for this site, but I have to reflect and say that having done similar types of data manipulation in Drupal 7, Drupal 9 makes this stuff much, much simpler!