Some Virtual Try-on (VTON) Research


阅读数: 次   
字数统计: 1.7k字   |   阅读时长: 10分

最近跟项目在做一些，虚拟试衣Virtual Try-on (VTON）的工作，记录一下调研的数据以及开源的论文以及模型。

Clothing dataset

Over 5,000 images of 20 different classes.

This dataset can be freely used for any purpose, including commercial:

For example:

  • Creating a tutorial or a course (free or paid)
  • Writing a book
  • Kaggle competitions (as an external dataset)
  • Training an internal model at any company

Data

The images.csv file contains:

  • image - the ID of the image (use it to load the image from images/<ID>.jpg)
  • sender_id - the ID of a person who contributed the image
  • label - the class of the image
  • kids - flag, True if it’s clothes for kids
1
https://www.kaggle.com/agrigorev/clothing-dataset-full/

Top-10 subset

Images of some classes don’t appear very often. Training a neural network to predict these classes is quite difficult — we need at least 100-200 images of each class to make a meaningful model.

That’s why, for educational purposes, we created a subset of the full dataset that covers only the top-10 classes.

Check it here: 

1
https://github.com/alexeygrigorev/clothing-dataset-small

Examples

1
https://www.kaggle.com/agrigorev/collage

Do you use this dataset somewhere? Please submit a PR with a link

Acknowledgements

We’d like to thank

  • Kenes Shangereyev and Tagias.com for helping with 3000 images

  • All the 32 people who contributed their images to the dataset via the forms:

    • Patricia Goldberg
  • 1
    https://www.linkedin.com/in/patricia-goldberg/

  • Everyone who supported the initiative by engaging with the announcements on social media

It wouldn’t be possible to collect this dataset without your help!

A curated list of awesome research papers, projects, code, dataset, workshops etc. related to virtual try-on (VTON).

Image-based (2D) Virtual Try-on

ACCV 2020

  • CloTH-VTON: Clothing Three-dimensional reconstruction for Hybrid image-based Virtual Try-ON - Project

    ECCV 2020

  • Do Not Mask What You Do Not Need to Mask: a Parser Free Virtual Try-On - Paper

    CVPR 2020

  • Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content - Paper/Code/Data

  • Image Based Virtual Try-On Network From Unpaired Data - Paper

  • Semantically Multi-modal Image Synthesis - Paper/Code/Model

    CVPRW 2020

  • CP-VTON+: Clothing Shape and Texture Preserving Image-Based Virtual Try-On - Paper/Code/Data/Model

  • 3D Reconstruction of Clothes using a Human Body Model and its Application to Image-based Virtual Try-On - Paper/Project

    ICCV 2019

  • VTNFP: An Image-Based Virtual Try-On Network With Body and Clothing Feature Preservation - Paper

  • ClothFlow: A Flow-Based Model for Clothed Person Generation - Paper

    ICCVW 2019

  • UVTON: UV Mapping to Consider the 3D Structure of a Human in Image-Based Virtual Try-On Network, Paper

  • LA-VITON: A Network for Looking-Attractive Virtual Try-On - Paper
  • Robust Cloth Warping via Multi-Scale Patch Adversarial Loss for Virtual Try-On Framework - Paper
  • Powering Virtual Try-On via Auxiliary Human Segmentation Learning - Paper

  • Generating High-Resolution Fashion Model Images Wearing Custom Outfits - Paper

    ECCV 2018

  • Toward Characteristic-Preserving Image-based Virtual Try-On Network - Paper, Code

  • SwapNet: Garment Transfer in Single View Images - Paper, Code (unofficial)

    CVPR 2018

  • VITON: An Image-based Virtual Try-on Network - Paper, Code/Model

    Others

  • Keypoints-Based 2D Virtual Try-on Network System, JAKO 2020 - Paper

  • Virtual Try-On With Generative Adversarial Networks: A Taxonomical Survey - Book chapter
  • LGVTON: A Landmark Guided Approach to Virtual Try-On - Paper
  • SieveNet: A Unified Framework for Robust Image-Based Virtual Try-On, WACV 2020 - Paper, Code/Model (unofficial)
  • GarmentGAN: Photo-realistic Adversarial Fashion Transfer - Paper
  • Toward Accurate and Realistic Virtual Try-on Through Shape Matching and Multiple Warps - Paper
  • FashionFit Analysis of Mapping 3D Pose and neural body fit for custom virtual try-on, IEEE Access 2020 - Paper
  • SP-VITON: shape-preserving image-based virtual try-on network, Multimedia Tools and Applications 2019 - Paper
  • VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss, Eurographics 2019 Posters - Paper, Code/Model
  • Image-Based Virtual Try-on Network with Structural Coherence, ICIP 2019 - Paper
  • End-to-End Learning of Geometric Deformations of Feature Maps for Virtual Try-On - Paper
  • M2E-Try On Net: Fashion from Model to Everyone - Paper

3D Virtual Try-on

ECCV 2020

  • BCNet: Learning Body and Cloth Shape from A Single Image - Paper, Code/Data
  • GAN-based Garment Generation Using Sewing Pattern Images - Paper/Code/Model/Data
  • Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images - Paper/Data
  • SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing - Paper/Code/Data

  • CLOTH3D: Clothed 3D Humans - Paper

    CVPR 2020

  • Learning to Transfer Texture from Clothing Images to 3D Humans - Paper/Code

  • TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style - Paper/Code/Data

  • Learning to Dress 3D People in Generative Clothing - Paper/Code/Data

    ICCV 2019

  • Multi-Garment Net: Learning to Dress 3D People from Images - Paper/Code/Data

  • 3DPeople: Modeling the Geometry of Dressed Humans - Paper

  • GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping - Paper/Data

    ECCV 2018

  • DeepWrinkles: Accurate and Realistic Clothing Modeling - Paper

    CVPR 2018

  • Video Based Reconstruction of 3D People Models - Paper/Code/Data

    Others

  • CloTH-VTON: Clothing Three-dimensional reconstruction for Hybrid image-based Virtual Try-ON, ACCV 2020 - Project

  • Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On, ACM SCA 2020 - Paper/Project
  • DeePSD: Automatic Deep Skinning And Pose Space Deformation For 3D Garment Animation - Paper
  • 3D Reconstruction of Clothes using a Human Body Model and its Application to Image-based Virtual Try-On, CVPRW 2020 - Paper/Project
  • Learning-Based Animation of Clothing for Virtual Try-On, Eurographics 2019 - Paper/Project
  • Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation, SIGGRAPH Asia 2019 - Paper/Code
  • 3D Virtual Garment Modeling from RGB Images, ISMAR 2019 - Paper
  • Deep Garment Image Matting for a Virtual Try-on System, ICCVW 2019 - Paper
  • Learning a Shared Shape Space for Multimodal Garment Design, SIGGRAPH Asia 2018 - Paper/Code/Data
  • Detailed Garment Recovery from a Single-View Image, ACM TOG 2018 - Paper
  • ClothCap: Seamless 4D Clothing Capture and Retargeting, SIGGRAPH 2017 - Paper
  • Virtual Try-On through Image-Based Rendering, IEEE T-VCG 2013 - Paper
  • Markerless Garment Capture, ACM TOG 2008 - Paper/Data

Multi-Pose Guided Virtual Try-on

  • Down to the Last Detail: Virtual Try-on with Detail Carving - Paper, Code/Model

  • Towards Multi-pose Guided Virtual Try-on Network, ICCV 2019 - Paper, Code

  • FIT-ME: IMAGE-BASED VIRTUAL TRY-ON WITH ARBITRARY POSES, ICIP 2019 - Paper

  • Virtually Trying on New Clothing with Arbitrary Poses, ACM MM 2019 - Paper

  • FashionOn: Semantic-guided Image-based Virtual Try-on with Detailed Human and Clothing Information, ACM MM 2019 - Paper

Video Virtual Try-on

  • FW-GAN: Flow-Navigated Warping GAN for Video Virtual Try-On, ICCV 2019 - Paper
  • Unsupervised Image-to-Video Clothing Transfer, ICCVW 2019 - Paper

Non-clothing Virtual Try-on

  • CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfer - Paper
  • Regularized Adversarial Training for Single-Shot Virtual Try-On, ICCVW 2019 - Paper
  • Disentangled Makeup Transfer with Generative Adversarial Network - Paper
  • PIVTONS: Pose Invariant Virtual Try-On Shoe with Conditional Image Completion, ACCV 2018 - Paper
  • Virtual Try-on of Eyeglasses using 3D Model of the Head - Paper
  • A MIXED REALITY SYSTEM FOR VIRTUAL GLASSES TRY-ON - Paper
  • A virtual try-on system in augmented reality using RGB-D cameras for footwear personalization - Paper

Pose-Guided Human Synthesis

  • PoNA: Pose-Guided Non-Local Attention for Human Pose Transfer, IEEE T-IP 2020 - Paper
  • Pose-Guided High-Resolution Appearance Transfer via Progressive Training - Paper
  • Recapture as You Want - Paper
  • Generating Person Images with Appearance-aware Pose Stylizer, IJCAI 2020 - Paper, Code
  • Controllable Person Image Synthesis with Attribute-Decomposed GAN, CVPR 2020 - Paper, Code
  • Deep Image Spatial Transformation for Person Image Generation, CVPR 2020 - Paper, Code
  • Neural Pose Transfer by Spatially Adaptive Instance Normalization, CVPR 2020 - Paper, Code
  • Guided Image-to-Image Translation with Bi-Directional Feature Transformation, ICCV 2019 - Paper, Code
  • Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis, ICCV 2019 - Paper, Code
  • ClothFlow: A Flow-Based Model for Clothed Person Generation, ICCV 2019 - Paper
  • Progressive Pose Attention for Person Image Generation, CVPR 2019 - Paper, Code
  • Dense Intrinsic Appearance Flow for Human Pose Transfer, CVPR 2019 - Paper, Code
  • Unsupervised Person Image Generation with Semantic Parsing Transformation, CVPR 2019, TPAMI 2020 - Paper, Code
  • Pose Guided Fashion Image Synthesis Using Deep Generative Model - Paper
  • Synthesizing Images of Humans in Unseen Poses, CVPR 2018 - Paper, Code
  • Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis, NeurIPS 2018 - Paper
  • Deformable GANs for Pose-based Human Image Generation, CVPR 2018 - Paper
  • Pose-Normalized Image Generation for Person Re-identification, ECCV 2018 - Paper
  • Disentangled Person Image Generation, CVPR 2018 - Paper/Code/Data
  • A Variational U-Net for Conditional Appearance and Shape Generation, CVPR 2018 - Paper, Code
  • Human Appearance Transfer, CVPR 2018 - Paper
  • Pose guided person image generation, NeurIPS 2017 - Paper, Code

Datasets for Virtual Try-on

Pull requests are welcome!

赏

谢谢你请我吃糖果

扫一扫，分享到微信

微信分享二维码
载入天数...载入时分秒...

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

计算机科学先驱阿兰·图灵说，“We can only see a short distance ahead, but we can see plenty there that needs to be done.”<br> 我们只能向前看到一小段未来，但我们可以在那里看到足够多可以做的事。很难说当代一个受过良好教育的人，一个理解银河系、理解宇宙演化史、理解现代社会法则的人和16世纪一个仍然相信地球是平的的人有什么本质的区别。看起来16世纪的你一定会比21世纪的你更愚蠢，但从未知的方面看，两者其实同等幼稚。今天宏大的宇宙观可能就是另一个版本的平坦地球，如果被25世纪的人观察的话。正因为如此，图灵的话才会有意义。正是因为所有人都没有大自然聪明，才会有文明的发展，才会有人秉持真理去挑战所谓的人类权威；正是因为问题永远都无法完善，才会有生机与希望，才会有足够多的工作可以做。 所以我想实际上图灵是在说，既然都离不开这艘飞船，那就让我们在这里做一些可以做的事吧，或许会有些惊喜也说不定呢。