how to convert dzi files to multi-tile pyramidal t

Question

In reference to the answer (how to convert dzi (deep zoom) files to full image)

I am a noob on this topic, please correct if I am wrong.

I am trying to create a multi-tiled tiff from .dzi (deep zoom image) format, how I am doing is:

(i) Pick the max level folder.

(ii) Vertically Stitch all the n columns of m rows (m_n.jpeg, I am referring to the images saved in this format) as png images. So they are occupying quite a considerable amount of space.

(iii) Finally, I horizontally merge all these vertical png images into a single full image png using pyvips.Image.write_to_file().

(iv) Finally using vips im_save, I convert the full image png to tiff.

My concern now is this process is taking almost 2 hours for it to make full image png for 30,000 base_tiles, and also it accounts to 10+ GB of size (Full Image png).

Is there any better and faster way to do the .dzi to tiff conversion?

Answer 1

libvips has an arrayjoin operator that can join a set of tiles together into a large image.

You can use it like this (on the linux command-line):

vips arrayjoin "$(ls *.jpeg | sort -t_ -k2g -k1g)" x.tif[tile,pyramid,compression=jpeg] --across 20

That will load all the JPG image in the current directory, assemble them into a huge grid, 20 images across, and write as a TIFF pyramid. You'd need to check the size of your grid, obviously, and adjust the across parameter.

The $() part sorts the filenames of the form x_y.jpg by y first, then x, in numeric order. Without that, the tiles will be transposed, annoyingly.

That's assuming overlap 0. If your tiles have an overlap, you'll need to set the hspacing and vspacing options to control how tiles are positioned. For example:

vips arrayjoin "$(ls *.jpg | sort -t_ -k2g -k1g)" x.tif --across 20 --hspacing 254 --vspacing 254

Will position the tiles every 254 pixels horizontally and vertically.

arrayjoin has to be able to open all of the input images, so it needs a lot of file descriptors. Most linuxes default to a maximum of 1024 files open at once per process, so you'll probably need to raise that number. Usually you just edit a couple of config files and log out and in again. I set my system to 65536, but you can use any number.

Windows has a hard limit of 2000 files per process which you cannot change. You'll need to assemble in sections on that platform.

Here's a worked example. First, create a deepzoom pyramid with no overlaps:

john@kiwi:~/pics/x$ vips dzsave ~/pics/k2.jpg x --overlap 0
john@kiwi:~/pics/x$ cd x_files/11
john@kiwi:~/pics/x/x_files/11$ ls
0_0.jpeg  0_7.jpeg  1_5.jpeg  2_3.jpeg  3_1.jpeg  3_8.jpeg  4_6.jpeg  5_4.jpeg
0_1.jpeg  0_8.jpeg  1_6.jpeg  2_4.jpeg  3_2.jpeg  4_0.jpeg  4_7.jpeg  5_5.jpeg
0_2.jpeg  1_0.jpeg  1_7.jpeg  2_5.jpeg  3_3.jpeg  4_1.jpeg  4_8.jpeg  5_6.jpeg
0_3.jpeg  1_1.jpeg  1_8.jpeg  2_6.jpeg  3_4.jpeg  4_2.jpeg  5_0.jpeg  5_7.jpeg
0_4.jpeg  1_2.jpeg  2_0.jpeg  2_7.jpeg  3_5.jpeg  4_3.jpeg  5_1.jpeg  5_8.jpeg
0_5.jpeg  1_3.jpeg  2_1.jpeg  2_8.jpeg  3_6.jpeg  4_4.jpeg  5_2.jpeg
0_6.jpeg  1_4.jpeg  2_2.jpeg  3_0.jpeg  3_7.jpeg  4_5.jpeg  5_3.jpeg

You can see it's made a grid of tiles 6 across and 9 down.

Now reassemble the tiles and write as a TIFF pyramid:

john@kiwi:~/pics/x/x_files/11$ vips arrayjoin "$(ls *.jpeg | sort -t_ -k2g -k1g)" x.tif[tile,pyramid,compression=jpeg] --across 6
john@kiwi:~/pics/x/x_files/11$ vipsheader x.tif 
x.tif: 1524x2286 uchar, 3 bands, srgb, tiffload_stream

With pyvips it would be something like:

#!/usr/bin/env python3

import pyvips

tiles_across = 142
tiles_down = 150

tiles = [pyvips.Image.new_from_file(f"{x}_{y}.jpeg", access="sequential")
         for y in range(tiles_down) for x in range(tiles_across)]
im = pyvips.Image.arrayjoin(tiles, across=tiles_across)

im.write_to_file("x.jpg")

That took about 10 minutes and 6gb of ram to join 21,000 tiles on this laptop.