Move files on remote system, only if destination d

I'm trying to write an Ansible role that moves a number of files on the remote system. I found a Stack Overflow post about how to do this, which essentially says "just use the command module with 'mv'". I have a single task defined with a with_items statement like this where each item in dirs is a dictionary with src and dest keys:

- name: Move directories
  command: mv {{ item.src }} {{ item.dest }}
  with_items: dirs

This is good and it works, but I run into problems if the destination directory already exists. I don't want to overwrite it, so I thought about trying to stat each dest directory first. I wanted to update the dirs variable with the stat info, but as far as I know, there isn't a good way to set or update variables once they're defined. So I used stat to get the info on each directory and then saved the data with register:

- name: Check if directories already exist
  stat: path={{ item.dest }}
  with_items: dirs
  register: dirs_stat

Is there a way to tie the registered stat info to the mv commands? This would be easy if it were a single directory. The looping is what makes this tricky. Is there a way to do this without unrolling this loop into two tasks per directory?

This is not the simplest solution by any means, but if you wanted to use Ansible and not "unroll":

---
- hosts: all
  vars:
    dirs:
      - src: /home/ubuntu/src/test/src1
        dest: /home/ubuntu/src/test/dest1
      - src: /home/ubuntu/src/test/src2
        dest: /home/ubuntu/src/test/dest2
  tasks:
    - stat:
        path: "{{item.dest}}"
      with_items: dirs
      register: dirs_stat
    - debug:
        msg: "should not copy {{ item.0.src }}"
      with_together:
        - dirs
        - dirs_stat.results
      when: item.1.stat.exists

Simply adapt the debug task to run the appropriate command task instead and when: to when: not ....