SMPLify-X is an open-source project that creates an expressive 3D body capture with detailed hands, face, and body from a single image. This project is a result of a research of 7 researchers and computer scientists at Max Planck ETH Center for Learning Systems.
The researchers train AI to rich dataset to provide detailed 3D reconstruction that recognize human body pose, hand gestures, and facial experssion. Unlike other methods which are using multiple cameras to reconstruct 3D model, SMPLify-X uses only one image as an input.
The team uses large dataset of 3D scans to produce expressive realistic model. They created a body pose prior that is based on a large dataset of thousands of body pose models.
The project uses PyTorch (an open source machine learning framework), and PyRender (a lightweight Python library for 3D rendering and visualization which is also open-source licensed under MIT license).
The Project also uses VPoser which is a variational human body pose prior by the same researchers. It's released also for free for non-commercial use.
- 2D body pose detector with support of body keypoints, hand keypoints, and facial landmarks.
- Using detailed collision-based model for meshes.
- Gender classification.
- EHF (Expressive Hand Face) dataset for evaluation.
- PyTorch implementation achieves a speedup of more than 8x over Chumpy.
The source code is released for free for non-commercial scientific research purposes. Contact firstname.lastname@example.org for commercial licensing (and all related questions for business applications).
The original images used for the figures 1 and 2 of the paper can be found in this link. The images in the paper are used under license from gettyimages.com. We have acquired the right to use them in the publication, but redistribution is not allowed. Please follow the instructions on the given link to acquire right of usage. Our results are obtained on the 483 × 724 pixels resolution of the original images.
Netron is an open-source multi-platform visualizer and editor for artificial intelligence models. It supports many extensions for deep learning, machine learning and neural network models. Netron is using Electron/ NodeJS and it has a binary application release for Windows, Linux and macOS. Netron is popular among data scientists, The project's...Read more...
A group of scientists from EPFL (École polytechnique fédérale de Lausanne) in Switzerland have developed a deep-learning based motion-capture software that uses multiple camera views to model the movements of a fly (Drosophila melanogaster) in three dimensions. The ultimate aim is to use this knowledge to design fly-like robots. In...Read more...
The increasing prevalence of antibiotic-resistant bacteria is a growing problem around the world. Every year, millions of people are infected with drug-resistant pathogens, and a lot of people die from pneumonia or bloodstream infections. In recent years, researchers have been working to make use of genome sequencing to identify antibiotic-resistant...Read more...
IBM Research lab in Zurich Open-Sourced three AI Projects to help accelerate cancer research. The first project named PaccMann (Prediction of anticancer compound sensitivity with Multi-modal attention-based neural networks), is a multimodal deep learning solution that uses data from disparate sources to help predict how cells in diseased tissue will...Read more...
Radiation Therapy is a widely used treatment modality for many types of cancer, but it has potential adverse effects to healthy surrounding organs. So, accurately identifying all Organs at Risk (OARs) is a very important step in Radiation Therapy Planning. However, manually doing so is time consuming and error-prone. A...Read more...