Generative Machine Learning GAN

Generative Machine Learning GAN

GAN

The dataset was web-scraped for an original 20k samples, then a custom MRCNN model was trained for image segmentation and cropping before being fed into the 128 DCGAN, trained on local hardware, 1660

Generative Model Course

GAN Course

GAN Project/Paper

Style GAN

Paper

GAN Image Superresolution

GAN

Style-GAN

Research

Research

  • Realless Generative webs with blinking eyes

Music Generation

Video Generation

  • Hyperchroma A music player that creates real-time music videos (hyperchroma.app)

Colorization

  • Pix2pix vs CycleGAN
    • Cycle-GAN can work in an ‘unpaired’ manner and various architectural differences. Unpaired image translation is much harder, as you demand the model to learn objects of different shapes, size, angle, texture, location in different scenes and settings on top of the actual task (coloring in this case). Requires more data and you don’t have fine control on the learning. Formulating the coloring problem as a paired task makes more sense as you simply decrease the complexity of the problem without increasing data collection/annotation work.
    • The whole point about using CycleGAN is that it can learn in unpaired situations. And it works well in the context of style transfer tasks where the changes are really bold and less nuanced. But, in the context of image colorization, the changes are really subtle and also there are way more options to choose colors than changing a horse to zebra. The other thing is that learning to change a colored image to black and white is much easier for the model than learning to colorize it which can lead to a bad learning procedure.
    • The most prominent differences is that CycleGAN helps when you have unpaired images and you want to go from one class to the other (Horse to Zebra for example) but in the Pix2Pix paper, the images that you get after the inference, are the input images but with some new features (black&white to colorized or day time to night time of a scene). In pix2pix, a conditional GAN (one generator and one discriminator) is used with some supervision from L1 loss. In CycleGAN, you need two generators and two discriminators to do the task: one pair for going from class A to B and one pair for going from class B to A. Also you need Cycle Consistency Loss to make sure that the models learn to undo the changes they make.
  • pix2pixhd is pix2pix in higher resolution
  • pix2pix tutorial and example:

Machine Learning: Image Generator

GAN

Generation

Lightboard


Last modified March 6, 2023: update (7eba5da)