Piwigo thumbnail generation script

Piwigo thumbnail generation script

I use Piwigo 1 photo gallery to host my own photos and I am happy with everything expect one thing, thumbnail generation! Every photo in my gallery has a set of thumbnails of different sizes

  • Square 120 x 120 pixels, crop
  • Thumbnail 144 x 144 pixels
  • XXS - tiny 240 x 240 pixels
  • XS - extra small 432 x 324 pixels
  • S - small 576 x 432 pixels
  • M - medium 792 x 594 pixels
  • L - large 1008 x 756 pixels
  • XL - extra large 1224 x 918 pixels
  • XXL - huge 1656 x 1242 pixels

I mostly use the first 6 thumbnail types and Piwigo creates these on demand, but I want to pre-generate all thumbnails for faster performance than wait the thumbnails to be generated when I need to view them for the first time. There is a function in Piwigo to do this, but it often times out for me and my ARM server takes 2 days to generate all thumbnails. I want to be able to generate the thumbnails on a faster computer and then just transfer the thumbnails to my server. So I created a small script to generate the needed thumbnails.

The script

The script uses ImageMagick to create the following sizes:

  • Square 120 x 120 pixels, crop
  • Thumbnail 144 x 144 pixels
  • XXS - tiny 240 x 240 pixels
  • S - small 576 x 432 pixels
  • M - medium 792 x 594 pixels

sourceDir points to the base directory where you store the full resolution photos and destDir points to Piwigo’s internal thumbnail cache directory. It then loops all jpegs found in the sourceDir, does some basic error checking and if the medium thumbnail does not exists in the destDir it will create a set of thumbnails. Therefore the script can be run daily or weekly on or off the server. For every time the script loops 100 photos it will print a status line and execution time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
#!/bin/bash
#Author: Poul Serek

shopt -s globstar

echo "Starting Piwigo thumbnail generation"

#Remember a trailing '/'
sourceDir="/mnt/usbdisk/photos/Presentation/"
destDir="/var/www/html/gallery/_data/i/galleries/"

counter=0
fnNoExt=""
fnExt=""
fnPath=""

STARTTIME=$(date +%s)

for file in "$sourceDir"/**/*.{jpg,JPG,jpeg,JPEG}
do
        if [[ ! -f "$file" ]]
        then
                continue
        fi

        fnNoExt="${file%.*}"
        fnExt="${file##*.}"
        fnPath="${file%/*}"
        fnPath="${fnPath#$sourceDir}"
        fnNoExt="${fnNoExt#$sourceDir}"

        echo "${fnNoExt}"

        mkdir -p "${destDir}${fnPath}"

        #Error checking
        result=$(jpeginfo -c "$file")
        if [[ $result != *"[OK]"* ]]
        then
                echo $result
        fi

        #If the medium thumbnail exists we assume that the rest also exists and skip this image
        if [ ! -f "${destDir}${fnNoExt}-me.${fnExt}" ]; then
                #echo "MISSING! ${destDir}${fnNoExt}-me.${fnExt}"
                #Store correctly oriented base image (medium) in memory. All other thumbnails are created from this
                convert "${file}" -auto-orient -resize 792x594 -write mpr:baseline +delete \
                mpr:baseline -write "${destDir}${fnNoExt}-me.${fnExt}" +delete \
                mpr:baseline -resize 144x144 -write "${destDir}${fnNoExt}-th.${fnExt}" +delete \
                mpr:baseline -resize 240x240 -write "${destDir}${fnNoExt}-2s.${fnExt}" +delete \
                mpr:baseline -resize 576x432 -write "${destDir}${fnNoExt}-sm.${fnExt}" +delete \
                mpr:baseline -define jpeg:size=144x144 -thumbnail 120x120^ -gravity center -extent 120x120 "${destDir}${fnNoExt}-sq.${fnExt}"
        fi
        counter=$[$counter +1]
        if [ $(($counter%100)) -eq 0 ]; then
                ENDTIME=$(date +%s)
                echo "Processed: ${counter} - Executing for $((($ENDTIME - $STARTTIME)/60)) minutes"
        fi
done

ENDTIME=$(date +%s)
echo "It took $((($ENDTIME - $STARTTIME)/60)) minutes to complete this task..."

Example of execution output

The script should be run as the same user as Piwigo is run, for me that is www-data.

$ sudo su -s /bin/bash www-data -c '/home/user/piwi.sh'
Starting Piwigo thumbnail generation
Processed: 100 - Executing for 18 minutes
Processed: 200 - Executing for 36 minutes
It took 38 minutes to complete this task...

And you should now be able to see the thumbnails in Piwigo

Quirks

Some of my photos seems to be stretched when viewing them after clicking on the picture in the gallery

Piwigo stretched image caused by an incorrect width and height attribute on the IMG tag

The thumbnails themselves seem to work fine when opening the thumbnail in a new browser tab. I am sure I must be doing something wrong in the thumbnail generation, but for now I just hacked Piwigo by removing the height and width attributes from themes/default/template/picture_content.tpl:

  1. Open themes/default/template/picture_content.tpl
  2. Delete {$current.selected_derivative->get_size_htm()} from the line beginning with <img
  3. Restart the webserver
Fixed the image by hacking Piwigo and removing the width and height attributes

The stretched thumbnails should now look as expected. Please leave me a comment if you have some questions of improvements to the script!

  1. Piwigo is an open source photo gallery using PHP and MySQL 

8 Comments

bob5972

I have a similar problem, but for a difference reason. I have a piwigo gallery hosted on a Linode where I don’t have enough disk space to host all my full-sized photos. Instead, I mount all the originals over sshfs from a machine with a slower network connection and more disk space, but generating the thumbnails can take 10-seconds a piece (as I re-upload the image each time). I’ve adapted your script so that I can generate all the thumbnails on my desktop, and then just push those to the server. (Saving the slow link for when someone actually tries to view the originals.)

Thanks for writing this up!

Poul Serek

Hi Bob

Glad you could use the script. I actually had a similar setup with a very slow sshfs connection to my full-size photos but finally moved everything to my local server.

/Poul

Olav

Thanks for the inspiration. I found the reason for the stretched pictures (I think), For each picture there is a value in the database that tells if the image is rotated or not. I assume this is used for setting the image attributes where the image is displayed. This is set when the thumbnail is generated, so if a thumbnail is generated by an external script and the image is rotated there will be a mismatch between images and database setting. I.e. the images will be in portrait mode, but the database thinks it is landscape. I solved this by letting Piwigo generate thumbnails for rotated images and this made all images look nice.

Poul Serek

Hi Olav

Thanks for the clue! It makes sense then that my script fails on these rotated pictures. One could expand the script and modify the picture rotation flag in the database, I might do this if I find the time.

/Poul

Stefan

Hi Poul,
Thank you for this piece of code, I found it very useful.
But I have some questions and observations.

  1. If I want to generate the entire set of images used by piwigo, I must start with biggest one, XXL - 1656 x 1242 pixels, isn’t it? And using this one as base to generate all others.

  2. As long as I am a begginer in ImageMagick, is it possible to substitute “jpeginfo -c” with some internal ImageMagick option/command ?

  3. Maybe it will be useful to include in each directory (by copying from convenient location?) an index.html file to deny unwanted access to that folder as per Piwigo practice.

Thank you,
Stefan

Poul Serek

Hi Stefan

1) In a perfect world you would generate all thumbnails from the original, but that takes time. You might save some time by generating the larger thumbnails from the original or the XXL and for the smaller ones you could consider generating them from M

2) The ImageMagick Identify command with the correct parameters can detect corrupt jpgs. This is taken from the ImageMagick.org site:

Exit Status
The identify program returns a non-zero exit status if a corrupted image is encountered and you add a Regard Warnings Control.

error=`identify -regard-warnings image 2>&1 >/dev/null;`
  if [ $? -eq 0 ]; then
    echo "The image is good"
  else
    echo "The image is corrupt or known format"
    echo "$error"
  fi

3) That could be useful, if I get the time I might try to add that feature to the script.

TeclisB

Hi there!
Thanks a lot for the script I ll try it as soon as my photos are sorted!
On you screenshot, you seems to use a plugin which enable kind of map with localisations of your photos.
I tried “RV Maps & Earth” and it’s a cool plugin but seems less interesting than yours. What’s your plugin?

Thanks again!

Leave a reply