Kategori: Uncategorized

Free Ebook: Machine Learning – A First Course for Engineers and Scientists

An interesting free ebook

When we developed the course Statistical Machine Learning for engineering students at Uppsala University, we found no appropriate textbook, so we ended up writing our own. It will be published by Cambridge University Press in 2021.

Andreas Lindholm, Niklas Wahlström, Fredrik Lindsten, and Thomas B. Schön

A draft of the book is available below. We will keep a PDF of the book freely available also after its publication.

Latest draft of the book (older versions >>)

Table of Contents

  1. Introduction
    • The machine learning problem
    • Machine learning concepts via examples
    • About this book
  2. Supervised machine learning: a first approach
    • Supervised machine learning
    • A distance-based method: k-NN
    • A rule-based method: Decision trees
  3. Basic parametric models for regression and classification
    • Linear regression
    • Classification and logistic regression
    • Polynomial regression and regularization
    • Generalized linear models
  4. Understanding, evaluating and improving the performance
    • Expected new data error: performance in production
    • Estimating the expected new data error
    • The training error–generalization gap decomposition
    • The bias-variance decomposition
    • Additional tools for evaluating binary classifiers
  5. Learning parametric models
    • Principles pf parametric modelling
    • Loss functions and likelihood-based models
    • Regularization
    • Parameter optimization
    • Optimization with large datasets
    • Hyperparameter optimization
  6. Neural networks and deep learning
    • The neural network model
    • Training a neural network
    • Convolutional neural networks
    • Dropout
  7. Ensemble methods: Bagging and boosting
    • Bagging
    • Random forests
    • Boosting and AdaBoost
    • Gradient boosting
  8. Nonlinear input transformations and kernels
    • Creating features by nonlinear input transformations
    • Kernel ridge regdression
    • Support vector regression
    • Kernel theory
    • Support vector classification
  9. The Bayesian approach and Gaussian processes
  10. Generative models and learning from unlabeled data
    • The Gaussian mixture model and discriminant analysis
    • Cluster analysis
    • Deep generative models
    • Representation learning and dimensionality reduction
  11. User aspects of machine learning
    • Defining the machine learning problem
    • Improving a machine learning model
    • What if we cannot collect more data?
    • Practical data issues
    • Can I trust my machine learning model?
  12. Ethics in machine learning (by David Sumpter)
    • Fairness and error functions
    • Misleading claims about performance
    • Limitations of training data

Speeding Up Reinforcement Learning with a New Physics Simulation Engine

Source: https://ai.googleblog.com/2021/07/speeding-up-reinforcement-learning-with.html

Reinforcement learning (RL) is a popular method for teaching robots to navigate and manipulate the physical world, which itself can be simplified and expressed as interactions between rigid bodies1 (i.e., solid physical objects that do not deform when a force is applied to them). In order to facilitate the collection of training data in a practical amount of time, RL usually leverages simulation, where approximations of any number of complex objects are composed of many rigid bodies connected by joints and powered by actuators. But this poses a challenge: it frequently takes millions to billions of simulation frames for an RL agent to become proficient at even simple tasks, such as walking, using tools, or assembling toy blocks.

NOAA Fisheries Steller Sea Lion Population Count

Original Competition: https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count

Best solutions:

Related Articles

Papers

Pengukuran Kualitas Ventilasi Dengan Sensor CO2

Belgia sudah mewajibkan pengukuran CO2 di tempat umum. [https://www.info-coronavirus.be/en/ventilation/]

Penelitian menganjurkan CO2 sebagai sarana mengukur resiko penularan COVID-19

Referensi

Popular Augmentation Library

Popular Augmentation library:

Augly

Augly (https://github.com/facebookresearch/AugLy) and Albumentations (https://github.com/albumentations-team/albumentations)

Augly example

Albumentations

Albumentations example:

Albumentations example

References

Covid-19 Detection with Chest Radiograph: Resources

Papers

Research Papers:

  • CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection – in this research, we present a method to generate synthetic chest X-ray (CXR) images by developing an Auxiliary Classifier Generative Adversarial Network (ACGAN) based model called CovidGAN. In addition, we demonstrate that the synthetic images produced from CovidGAN can be utilized to enhance the performance of CNN for COVID-19 detection. Classification using CNN alone yielded 85% accuracy. By adding synthetic images produced by CovidGAN, the accuracy increased to 95%. We hope this method will speed up COVID-19 detection and lead to more robust systems of radiology.

  • Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-rays – The best performing models are iteratively pruned to reduce complexity and improve memory efficiency. The predictions of the best-performing pruned models are combined through different ensemble strategies to improve classification performance. Empirical evaluations demonstrate that the weighted average of the best-performing pruned models significantly improves performance resulting in an accuracy of 99.01% and area under the curve of 0.9972 in detecting COVID-19 findings on CXRs. The combined use of modality-specific knowledge transfer, iterative model pruning, and ensemble learning resulted in improved predictions. We expect that this model can be quickly adopted for COVID-19 screening using chest radiographs.

  • COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs – Using these techniques, we showed the state of the art results on the open-access COVID-19 dataset. This work presents a 3-step technique to fine-tune a pre-trained ResNet-50 architecture to improve model performance and reduce training time. We call it COVIDResNet. This is achieved through progressively re-sizing of input images to 128x128x3, 224x224x3, and 229x229x3 pixels and fine-tuning the network at each stage. This approach along with the automatic learning rate selection enabled us to achieve the state of the art accuracy of 96.23% (on all the classes) on the COVIDx dataset with only 41 epochs. This work presented a computationally efficient and highly accurate model for multi-class classification of three different infection types from along with Normal individuals. This model can help in the early screening of COVID19 cases and help reduce the burden on healthcare systems.

More Papers (from https://www.kaggle.com/c/siim-covid19-detection/discussion/240838)

Source: https://pymed.ai/blog/researches-about-covid-19-chest-x-ray-classification-and-segmentation-with-deep-learning/

X-rays COVID-19 Localization

  1. COVID-19 detection from scarce chest x-ray image data using few-shot deep learning approach
  2. Identification of Images of COVID-19 from Chest X-rays Using Deep Learning: Comparing COGNEX VisionPro Deep Learning 1.0™ Software with Open Source Convolutional Neural Networks
  3. COVID-19 Image Data Collection
  4. CovidCTNet: an open-source deep learning approach to diagnose covid-19 using small cohort of CT images

Common Pitfalls (https://www.kaggle.com/c/siim-covid19-detection/discussion/240639)

Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans

The above is a link to a peer-reviwed paper recently published in the prestigious journal Nature Machine Intelligence. They examined 2,212 studies published in 2020, of which 415 were included after initial screening and, after quality screening, 62 studies were included in the systematic review.

Unfortunately they found that “…none of the models identified are of potential clinical use due to methodological flaws and/or underlying biases.“

Their main findings were:

  • Duplication and quality issues: Source issues, Frankenstein datasets, Implicit biases in the source data
  • Methodology issues: “…Diagnostic studies commonly compare their models’ performance to that of RT–PCR. However, as the ground-truth labels are often determined by RT–PCR, there is no way to measure whether a model outperforms RT–PCR from accuracy, sensitivity or specificity metrics alone. Ideally, models should aim to match clinicians using all available clinical and radiomic data…”

They then proceed to suggest a number of recommendations. The paper is well worth reading and is Open Access:

Top Solutions

This is an object detection and classification problem.

VinBigData Chest X-ray Abnormalities Detection

Components: Detection models (YOLO-V4 ), Specialized detector for aortic enlargement, Multi-label classifier-based post-processing. Image size 1280.

  • 4th place by @hiraiitsuki

    • model: yolov5x
    • image size: 640
    • TTA: 3 scale patterns and horizontal flip
    • ensemble: (4fold cv * 3 different preprocessed labels ) = 12 models

RSNA Pneumonia Detection Challenge

  • 1st place + code by @vaillant

    • classification-detection pipeline
    • detection: RetinaNet, Deformable R-FCN, Deformable Relation Networks.
    • classification: InceptionResNetV2 , Xception , DenseNet169.
    • Boxes were ensembled using: https://github.com/ahrnbom/ensemble-objdet

  • 2nd place by @dmytropoplavskiy

    • base model: custom RetinaNet (se-resnext101)
    • 512×512 resolution
    • augmentations: Mild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes.
    • ensemble: NMS

  • 3rd place + code by @pmcheng

    • base models: RetinaNet (resnet-50 and resnet-101 ) + focal loss
    • 224 x 224 resolution as a abdominal radiologist he considered that high image resolution was not necessary for pneumonia bounding box prediction.
    • augmentations: rotation, translation, scaling, and horizontal flipping + random constants
    • NMS to eliminate any overlapping bounding boxes

SIIM-ACR Pneumothorax Segmentation

  • 1st place + code by @sneddy

    • base models: AlbuNet (resnet34) , Resnet50 , SCSEUnet (seresnext50)
    • Combo loss: combinations of BCE, dice and focal.
    • start with 512×512 and uptrain on size 1024×1024
    • small batch (2-4) size without accumulation
    • detailed tricks on his summary and github

  • 2nd place + code by @lanjunyelan

    • classification and segmentation pipeline.

    • Classification: classify whether an image in related with pneumothorax or not. Multi-task model based on UNET (seresnext 50, seresnext101, efficientnet-b3
      ) with a branch for classifying. BCE + focal loss. Basic augmentation: hflip, scale, rotate, bright, blur

    • Segmentation: 2 base models: unet and deeplabv3. Loss: dice loss. Augmentation: same as classification

  • 3rd place + code by @bestfitting

    • base model UNET (resnet34, se-resnext50)
    • cropped lungs, 576×576 cropped images (1024×1024 initially).
    • Attention: CBAM
    • Loss: Lovasz Los.
    • No classification model, No classification loss

RSNA STR Pulmonary Embolism Detection

Segmentation

  • Unveiling COVID-19 from Chest X-ray with deep learning: a hurdles race with small data Enzo Tartaglione, Carlo Alberto Barbano, Claudio Berzovini, Marco Calandri, Marco Grangetto
    https://arxiv.org/abs/2004.05405
  • A Critic Evaluation of Methods for COVID-19 Automatic Detection from X-Ray Images Gianluca Maguolo, Loris Nanni, https://arxiv.org/abs/2004.12823

Resources :

Research Papers

  1. A Deep Neural Network to Distinguish COVID-19 from other Chest Diseases using X-ray Images ( https://www.mdpi.com/1424-8220/21/2/455/htm)
  2. COVID-CT-MD: COVID-19 Computed Tomography (CT) Scan Dataset Applicable in Machine Learning and Deep Learning (https://paperswithcode.com/paper/covid-ct-md-covid-19-computed-tomography-ct)
  3. CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection (https://ieeexplore.ieee.org/document/9093842)
  4. Synthesis of COVID-19 Chest X-rays using Unpaired Image-to-Image Translation 🙁https://paperswithcode.com/paper/synthesis-of-covid-19-chest-x-rays-using)