Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Style Transfer Generative Adversarial Networks: Learning To Play Chess Differently. “MakeupGAN: Makeup Transfer via Cycle-Consistent Adversarial Networks.” Computer Vision and Pattern Recognition, 2018. Xin Wang, Yujia Luo, Daniel Crankshaw, Alexey Tumanov, Fisher Yu, Joseph E.
PyTorch project page document
Torch implementation for understanding an image-to-image interpretation (i.y. pics2pix)withoutinput-output sets, for example:
Unpaired Image-to-Image Translation making use of Cycle-Consistent Adversarial Systems
Jun-Yan Zhu., Taesung Park., Phillip Isola, Alexei A new. Efros
Berkeley AI Analysis Lab, UC Berkeley
In ICCV 2017. (. equivalent advantages)
Jun-Yan Zhu., Taesung Park., Phillip Isola, Alexei A new. Efros
Berkeley AI Analysis Lab, UC Berkeley
In ICCV 2017. (. equivalent advantages)
This bundle consists of CycleGAN, pics2pix, simply because properly as some other methods like BiGAN/ALI and Apple company's paper H+U learning.
The program code was created by Jun-Yan Zhu and Taesung Park.
Update: Make sure you check out out PyTorch execution for CycleGAN and pics2pix.The PyTorch edition can be under active development and can produce results comparable or much better than this Flashlight version.
The program code was created by Jun-Yan Zhu and Taesung Park.
Update: Make sure you check out out PyTorch execution for CycleGAN and pics2pix.The PyTorch edition can be under active development and can produce results comparable or much better than this Flashlight version.
Some other implementations:
Tensorflow (by Harry Yang),Tensorflow (by Archit Rathore),Tensorflow (by Vehicle Huy),Tensorflow (by Xiaowei Hu), Tensorflow-simple (by Zhenliang He), TensorLayer (by luoxier),Chainer (by Yanghua Jin),Minimal PyTorch (by yunjey),Mxnet (by Ldpe2H),lasagne/Keras (by tjwei), Keras (by Simon Karlsson)
Applications
Monet Artwork to Photos
Selection Style Transfer
Object Transfiguration
Season Transfer
Picture Improvement: Thin depth of industry
Requirements
- Linux or OSX
- NVIDIA GPU + CUDA CuDNN (CPU setting and CUDA without CuDNN may work with minimal changes, but untested)
- For MAC customers, you need the Linux/GNU commands
gfind
andgwc
, which can be installed withmake install findutils coreutils
.
Getting Began
Set up
- Install torch and dependencies from https://github.com/flashlight/distro
- Install flashlight deals
nngraph
,course
,screen
- Clone this repo:
Apply a Pre-trained Design
- Down load the check photos (used by Alexei Efros):
- Download the pre-trained model
stylecezanne
(For Processor model, make use ofstylecezannecpu
):
- Today, allow's generate John Chemicalézanne design pictures:
The check outcomes will be saved to
Please be sure to relate to Design Zoo for even more pre-trained models.
./outcomes/stylecezannepretrained/latesttest/index.html
.Please be sure to relate to Design Zoo for even more pre-trained models.
./good examples/testvangoghstyleonaephotos.sh
can be an illustration screenplay that downloads the pretrained Vehicle Gogh design network and runs it on Efros'h pictures.Teach
- Download a dataset (e.gary the gadget guy. zebra and equine pictures from ImageNet):
- Teach a design:
- (CPU just) The exact same training command word without making use of a GPU or CUDNN. Establishing the environment variables
gpu=0 cudnn=0
pushes CPU only
- (Optionally) start the display machine to watch results as the design locomotives. (Notice Display UI for more information):
Test
- Finally, check the model:
The test results will become rescued to an HTML file right here:
./outcomes/horse2zebramodel/latesttest/index.html
.Design Zoo
Down load the pre-trained models with the pursuing software. The design will become saved to
./checkpoints/modelname/latestnetG.capital t7
.tangerine2apple
(orange -gt; apple company) andapple2orange
: qualified on ImageNet classesapple company
andred
.horse2zebra
(horse -gt; zebra) andzebra2horse
(zebra -gt; equine): trained on ImageNet categorieshorse
andzebra
.stylemonet
(scenery photograph -gt; Monet painting style),stylevangogh
(panorama image -gt; Truck Gogh artwork style),styleukiyoe
(landscape photo -gt; Ukiyo-e artwork style),stylecezanne
(panorama photograph -gt; Cezanne painting style): qualified on artwork and Flickr landscaping pictures.monet2image
(Monet artwork -gt; actual panorama): qualified on works of art and Flickr scenery photographs.cityscapesphoto2brand
(street picture -gt; tag) andcityscapeslabel2photo
(label -gt; road scene): qualified on the Cityscapes dataset.chart2sat
(chart -gt; aerial photo) andsitting2map
(aerial image -gt; map): educated on Google road directions.iphone2dslrflower
(iPhone pictures of blooms -gt; DSLR pictures of plants): educated on Flickr photos.
CPU models can become downloaded making use of:
, where
lt;namegt;
can end up beinghorse2zebra
,stylemonet
, etc. You simply need to appendcentral processing unit
to the target design.Teaching and Test Details
To train a model,
Models are saved to
See
./checkpoints/exptname
(can end up being changed by transferringcheckpointdir=yourdir
in train.lua).See
opttrain
inchoices.lua
for additional training choices.To test the model,
This will run the design named
A webpage with result images will end up being preserved to
Notice
exptname
in both directions on all pictures in/route/to/data/testA
and/route/to/data/testB
.A webpage with result images will end up being preserved to
./results/exptname
(can become changed by passingresultsdir=yourdir
in test.lua).Notice
opttest
inchoices.lua
for extra test options. Please usedesign=onedirectiontest
if you just would including to produce outputs of the qualified network in just one direction, and stipulatewhichdirection=AtoB
orwhichdirection=BtoA
to fixed the direction.There are other choices that can be utilized. For instance, you can indicate
resizeorcrop=plants
option to prevent resizing the image to squares. This is definitely indeed how we educated GTA2Cityscapes design in the projet web page and Cycada model. We ready the images at 1024pback button resolution, and usedresizeorcrop=crop fineSize=360
to work with the cropped images of dimension 360x360. We furthermore usedlambdaidentity=1.0
.Datasets
Download the datasets using the subsequent script. Numerous of the datasets had been collected by various other researchers. Please cite their papers if you make use of the data.
facades
: 400 pictures from the CMP Facades dataset. Quotationcityscapes
: 2975 images from the Cityscapes training place. Citationroutes
: 1096 training pictures scraped from Google Road directions.horse2zebra
: 939 equine images and 1177 zebra pictures down loaded from ImageNet making use of the keywordscrazy equine
andzebra
apple2orange
: 996 apple company images and 1020 tangerine images downloaded from ImageNet using the keywordsapple
andnavel orange
.summer season2winteryosemite
: 1273 summer Yosemite images and 854 winter Yosemite images had been downloaded making use of Flickr API. Find more details in our paper.monet2picture
,vangogh2photograph
,ukiyoe2picture
,cezanne2picture
: The art images had been downloaded from Wikiart. The real photos are usually downloaded from Flickr making use of the mixture of the labelslandscapingandlandscapephotography. The training established dimension of each course can be Monet:1074, Cezanne:584, Vehicle Gogh:401, Ukiyo-e:1433, Photographs:6853.iphone2dslrflower
: both courses of images had been downloaded from Flickr. The training established size of each course will be iPhone:1813, DSLR:3316. Find more details in our paper.
Screen UI
Optionally, for showing images during training and check, make use of the screen package.
- Install it with:
luarocks set up https://raw.githubusercontent.com/szym/screen/master/display-scm-0.rockspec
- After that begin the server with:
th -ldisplay.start
- Open up this Web link in your browser: http://localhost:8000
By default, the server listens on localhost. Pass
0.0.0.0
to permit external connections on any user interface:After that open
http://(hostname):(port)/
in your internet browser to load the remote control desktop computer.Setup Exercising and Check data
To train CycleGAN design on your personal datasets, you require to produce a data folder with two subdirectories
trainA
andtrainB
that consist of images from domain A new and N. You can test your design on your training fixed by establishingstage='train'
incheck.lua
. You can furthermore make subdirectoriestestA
andtestB
if you possess test information.You shouldnotanticipate our method to work on simply any arbitrary combination of input and output datasets (y.g.
catslt;-gt;key-boards
). From our experiments, we discover it works much better if two datasets reveal similar visible content material. For example,surroundings paintinglt;-gt;landscape photos
functions much better thanportrait artwork lt;-gt; surroundings photos
.zebraslt;-gt;horses
accomplishes compelling results whilecatslt;-gt;dogs
totally fails. Discover the following area for even more dialogue.Failure cases
Our model does not work properly when the check image is certainly rather different from the pictures on which the model is qualified, as is the situation in the body to the left (we qualified on race horses and zebras without riders, but test right here one a equine with a driver). See additional common failure instances right here. On translation jobs that include colour and texture adjustments, like several of those documented above, the method often succeeds. We possess also explored duties that need geometric modifications, with little success. For instance, on the task of
doglt;-gt;kitty
transfiguration, the discovered translation degenerates into making minimal changes to the input. We also see a lingering distance between the outcomes possible with paired training information and those accomplished by our unpaired method. In some cases, this gap may end up being very hard - or also impossible,- to near: for example, our technique occasionally permutes the brands for sapling and building in the output of the cityscapes photos-gt;labels job.Quotation
If you make use of this code for your research, please refer to our paper:
Related Projects:
pix2pix: Image-to-image interpretation making use of conditional adversarial netting
iGAN: Interactive Picture Generation via Generative Adversarial Systems
iGAN: Interactive Picture Generation via Generative Adversarial Systems
Cat Paper Collection
If you enjoy cats and kittens, and like reading cool graphics, vision, and ML papers, please verify out the Cat Paper Collection.
Acknowledgments
Program code borrows from pics2pix and DCGAN. The data loader can be modified from DCGAN and Context-Encoder. The generative network is followed from neural-style with Instance Normalization.