Skip to content
All projects

CESI · 5th year · team project

Deep learning: image processing

Three neural networks to automate image processing: classification, denoising and automatic captioning.

CompletedMay 2026 – Jun 2026
Deep learning: image processing

A team project by four students (FISA Computer Science, 5th year) for a fictitious company, TouNum, which digitises documents for its clients. The goal: automate its image processing with three deep learning models, built with TensorFlow in a Docker container with GPU access.

Deliverable 1 · Image classification

  • Dataset of 41,402 images in five classes (paintings, photos, diagrams, sketches, scanned text), with detection and repair of corrupted files
  • CNN with four convolutional blocks (32 to 256 filters, batch normalisation, dropout), trained with Adam, early stopping and learning-rate reduction
  • “Photo or not” task: 92.6% accuracy, AUC of 0.974 and F1 of 0.848 on the test set
  • Like-for-like comparison with two external architectures (fakv8 and fakv13): 93.7% and 94.7% validation accuracy, versus 92.4% for our CNN
Sample images from each of the five classes
Sample images from each of the five classes
Five-class confusion matrices: our CNN (87.2%) and fakv8 (92.4%)
Five-class confusion matrices: our CNN (87.2%) and fakv8 (92.4%)

Deliverable 2 · Image denoising

  • Starting question: can a network learn to denoise without clean images? Answered with the “Noise-as-Target” strategy (Noise2Noise): the network learns to recover an image from an even noisier version of it (Gaussian noise, σ = 0.04)
  • Residual U-Net with skip connections, trained with a combined loss: 0.3 × MAE + 0.7 × (1 − SSIM)
  • On the validation set: PSNR of 33.6 dB and SSIM of 0.925
  • Comparison with Citronn v1, a compact U-Net by a team member trained in a supervised way: 30.4 dB and 0.872
  • Graphical interface to try the denoiser on any image
Noisy image, image denoised by our U-Net (with PSNR and SSIM) and the original image
Noisy image, image denoised by our U-Net (with PSNR and SSIM) and the original image

Deliverable 3 · Automatic image captioning

  • COCO 2017 dataset: 118,287 images and about 590,000 captions, 7,000-word vocabulary, 80 / 10 / 10 split made per image to avoid any data leakage
  • Pre-trained, frozen InceptionV3 encoder, whose features are extracted once and cached on disk
  • Two decoders compared: a GRU with Bahdanau attention and a Transformer
  • Measured with the BLEU score, the Transformer beats the RNN (BLEU-4 of 13.7% versus 9.1% with greedy search); beam search (k = 5) clearly improves both models: 33.0% and 24.3%
  • Effect of data size: comparison with training on only 6,000 images (BLEU-4 of 5.5% and 6.4%)
Model attention map: the regions looked at for each word of the caption “a large passenger jet sitting on top of an airport runway”
Model attention map: the regions looked at for each word of the caption “a large passenger jet sitting on top of an airport runway”
BLEU-1 and BLEU-4 scores: greedy and beam search (200 images evaluated with greedy search, 50 with beam search)
BLEU-1 and BLEU-4 scores: greedy and beam search (200 images evaluated with greedy search, 50 with beam search)

The three models are the stages of a single image-processing pipeline: denoise, sort, then describe.