intel/BigDL

📦 Highlights

• *Note:* BigDL v2.5.0b1 has been updated to include functional and security updates. Users should update to the latest version.

📦 Highlights

• *Note:* BigDL v2.4.0 has been updated to include functional and security updates. Users should update to the latest version.

📦 Highlights

• *Note:* BigDL v2.3.0 has been updated to include functional and security updates. Users should update to the latest version.
• Nano
• Enhanced `trace` and `quantization` process (for PyTorch and TensorFlow model optimizations)
• New inference optimization methods (including Intel ARC series GPU support, CPU fp16, JIT int8, etc.)
• New inference/training features (including TorchCCL support, async inference pipeline, compressed model saving, automatic channels_last_3d, multi-instance training for customized TF train loop, etc.)
• Performance enhancement and overhead reduction for inference optimized model
• More user-friendly document and API design
• Orca:
• + 15 more

📦 Highlights

• *Note:* BigDL v2.2.0 has been updated to include functional and security updates. Users should update to the latest version.
• Nano
• Extend BigDL Nano inference to support iGPU and more data types (INT8/BF16/FP16 quantization)
• More performance features (e.g., InferenceOptimizer for Keras, Nano decorator for PyTorch training loop, Nano Context Manager for thread number control and autocast, etc.)
• Support installation with more PyTorch/TensorFlow versions and conditional dependencies on different platforms
• PPML
• Upgrade BigDL PPML solution to support new LibOS (e.g., Gramine1.3.1, Occlum0.29.2) with better security, higher performance, more stability and easier deployment.
• Support more Big Data frameworks (Spark 3.1.3, Flink, Hive etc.), more Python and Data Science tools (Numpy, Pandas, sklearn, Torch Serv, Triton, Flask etc.), and distributed DL training using Orca
• + 18 more

📦 Highlights

• *Note:* BigDL v2.1.0 has been updated to include functional and security updates. Users should update to the latest version.
• Orca
• Improve user experience and API consistency for Orca Estimators.
• Support directly save and load TensorFlow model format in Orca TensorFlow2 Estimator.
• Provide more examples (e.g. PyTorch brain image segmentation, XShards tutorials for distributed Python data processing), etc.
• Support customized metrics in Orca PyTorch Estimator.
• Nano
• New inference optimization pipelines, with more optimization methods and a new InferenceOptimizer
• + 26 more

📦 Highlights

• *Note:* BigDL v2.0.0 has been updated to include functional and security updates. Users should update to the latest version.

📦 Highlights

• Continue RNN optimization. We support both LSTM and GRU integration with MKL-DNN which acheives ~3x performance
• ONNX support. We support loading third party framework models via ONNX
• Richer data preprocssing support and segmentation inference pipeline support

📦 Details

• [New Feature] Full MaskRCNN model support with data processing
• [New Feature] Support variable-size Resize
• [New Feature] Support batch input for region proposal
• [New Feature] Support samples of different size in one minibatch
• [New Feature] MAP validation method implementation
• [New Feature] ROILabel enhancement to support both object detection and segmentation
• [New Feature] Grey image support for segmentation
• [New Feature] Add TopBlocks support for Feature Pyramid Networks (FPN)
• + 33 more

📦 Highlights

• Continue VNNI acceleration support, we add optimization for more CNN models including object detection models, enhance model scales generation support for VNNI.
• Add attention based model support, we add Transformer implementation for both lanuage model and translation model.
• RNN optimization, We support LSTM integration with MKL-DNN which acheives ~3x performance speedup.

📦 Details

• [New Feature] Add attention layer support
• [New Feature] Add FeedForwardNetwork layer support
• [New Feature] Add ExpandSize layer support
• [New Feature] Add TableOperation layer to support table calculation with different input sizes
• [New Feature] Add LayerNormalizaiton layer support
• [New Feature] Add Transformer support for both language and translation models
• [New Feature] Add beam search support in Transformer model
• [New Feature] Add Layer-wise adaptve rate scaling optim method
• + 24 more

📦 Highlights

• Add MKL-DNN Int8 support, especially for VNNI acceleration support. Low precision inference accelerates both latency and throughput significantly
• Add support for runnning MKL-BLAS models under MKL-DNN. We leverage MKL-DNN to speed up both training and inference for MKL-BLAS models
• Add Spark 2.4 support. Our examples and APIs are fully compatible with Spark 2.4, we released the binary for Spark 2.4 together with other Spark versions

📦 Details

• [New Feature] Add MKL-DNN Int8 support, especially for VNNI support
• [New Feature] Add support for runnning MKL-BLAS models under MKL-DNN
• [New Feature] Add Spark 2.4 support
• [New Feature] Add auto fusion to speed up model inference
• [New Feature] Memoery reorder support for low precision inference
• [New Feature] Add bytes support for DNN Tensor
• [New Feature] Add SAME padding in MKL-DNN layers
• [New Feature] Add combined (add/or) triggers for training completion
• + 16 more

📦 Highlights

• MKL-DNN support enhancement, which includes training optimization, more models training support and model serialization support
• A new distributed optimizer for models powered by MKL-DNN. This optimizer can overlap training and communication during the distributed training, which lead to a better scalability on multi-nodes

📦 Details

• [New Feature] A new optim method ParallelAdam which leverages the multi-thread capacity
• [New Feature] Add new validation methods HitRate, which is widely used in recommendation
• [New Feature] Add new validation methods NDCG, which is widely used in recommendation
• [New Feature] Support communication priority when synchronize parameter in the distributed training
• [New Feature] Support ModelBroadcast customization
• [New Feature] Add a new distributed optimizer for models powered by MKL-DNN. This optimizer can overlap training and communication during the distributed training, which lead to a better scalability on multi-nodes
• [API Change] Add batch size into the Python model.predict API
• [Enhancement] Add MKL-DNN training example for LeNet
• + 22 more

📦 Highlights

• We integrate [MKL-DNN](https://github.com/intel/mkl-dnn) as an alternative execution engine for CNN models. MKL-DNN provides better training/inference performance and less memory consuming. On some CNN models, we find there’s 2x throughput improvement in our experiment.
• Support using different optimization methods to optimize different parts of the model. This is necessary when train some models.
• Spark 2.3 support. We have tested our code and examples on Spark 2.3. We release the binary for Spark 2.3, and Spark 1.5 will not be supported.

📦 Details

• [New Feature] MKL-DNN integration. We integrate [MKL-DNN](https://github.com/intel/mkl-dnn) as an alternative execution engine for CNN models. It supports speedup layers like: AvgPooling, MaxPooling, CAddTable, LRN, JoinTable, Linear, ReLU, SpatialConvolution, SpatialBatchnormalization, Softmax. MKL-DNN provides better training/inference performance and less memory consuming.
• [New Feature] Layer fusion. Support layer fusion on conv + relu, batchnorm + relu, conv + batchnorm and conv + sum(some of the fusion can only be applied in the inference). Layer fusion provides better performance especially on inference. Currently layer fusion are only available for MKL-DNN related layers.
• [New Feature] Multiple optimization method support in optimizer. Support using different optimization methods to optimize different parts of the model.
• [New Feature] Add a new optimization method Ftrl, which is often used in recommendation model training.
• [New Feature] Add a new example: Training Resnet50 on ImageNet dataset.
• [New Feature] Add new OpenCV based image preprocessing transformer ChannelScaledNormalizer.
• [New Feature] Add new OpenCV based image preprocessing transformer RandomAlterAspect.
• [New Feature] Add new OpenCV based image preprocessing transformer RandomCropper.
• + 29 more

📦 Highlights

• Bring in a Keras-like API(Scala and Python). User can easily run their Keras code (training and inference) on Apache Spark through BigDL. For more details, see [this link](https://bigdl-project.github.io/0.5.0/#KerasStyleAPIGuide/keras-api-python/).
• Support load Tensorflow dynamic models(e.g. LSTM, RNN) in BigDL and support more Tensorflow operations, see [this page](https://bigdl-project.github.io/0.5.0/#APIGuide/tensorflow_ops_list/ ).
• Support combining data preprocessing and neural network layers in the same model (to make model deployment easy )
• Speedup various modules in BigDL (BCECriterion, rmsprop, LeakyRelu, etc.)
• Add DataFrame-based image reader and transformer

✨ New Features

• Tensor can be converted to OpenCVMat
• Bring in a new Keras-like API for scala and python
• Support load Tensorflow dynamic models(e.g. LSTM, RNN)
• Support load more Tensorflow operations(InvertPermutation, ConcatOffset, Exit, NextIteration, Enter, RefEnter, LoopCond, ControlTrigger, TensorArrayV3,TensorArrayGradV3, TensorArrayGatherV3, TensorArrayScatterV3, TensorArrayConcatV3, TensorArraySplitV3, TensorArrayReadV3, TensorArrayWriteV3, TensorArraySizeV3, StackPopV2, StackPop, StackPushV2, StackPush, StackV2, Stack)
• ResizeBilinear support NCHW
• ImageFrame support load Hadoop sequence file
• ImageFrame support gray image
• Add Kv2Tensor Operation(Scala)
• + 26 more

📦 Enhancement

• ImageFrame support serialization
• A default implementation of zeroGradParameter is added to AbstractModule
• Improve the style of the document website
• Models in different threads share weights in model training
• Speed up leaky relu
• Speed up Rmsprop
• Speed up BCECriterion
• Support Calling Java Function in Python Executor and ModelBroadcast in Python
• + 9 more

📋 API change

• Move DataFrame-based API to dlframe package
• Refine the Container hierarchy. The add method(used in Sequential, Concat…) is moved to a subclass DynamicContainer
• Refine the serialization code hierarchy
• Dynamic Graph has been an internal class which is only used to run tensorflow models
• Operation is not allowed to use outside Graph
• The getParamter method as final and private[bigdl], which should be only used in model training
• remove the updateParameter method, which is only used in internal test
• Some Tensorflow related operations are marked as internal, which should be only used when running Tensorflow models

🐛 Bug Fix

• Fix Sparse sample batch bug. It should add another dimension instead of concat the original tensor
• Fix some activation or layers don’t work in TimeDistributed and RnnCell
• Fix a bug in SparseTensor resize method
• Fix a bug when convert SparseTensor to DenseTensor
• Fix a bug in SpatialFullConvolution
• Fix a bug in Cosine equal method
• Fix optimization state mess up when call optimizer.optimize() multiple times
• Fix a bug in Recurrent forward after invoking reset
• + 18 more

📦 Highlights

• Supported all Keras layers, and support Keras 1.2.2 model loading. See [keras-support](https://bigdl-project.github.io/0.4.0/#ProgrammingGuide/keras-support/) for detail
• Python 3.6 support
• OpenCV support, and add a dozen of image transformer based on OpenCV
• More layers/operations

✨ New Features

• Models & Layers & Operations & Loss function
• + Add Criterion for keras: PoissonCriterion, KullbackLeiblerDivergenceCriterion, MeanAbsolutePercentageCriterion, MeanSquaredLogarithmicCriterion, CosineProximityCriterion
• + Support NHWC for LRN and BatchNormalization
• + Add LookupTableSparse (lookup table for multivalue)
• + Add activation argument for recurrent layers
• + Add MultiRNNCell
• + Add SpatialSeparableConvolution
• + Add MSRA filler
• + 18 more

📦 Enhancement

• Speed up BatchNormalization.
• Speed up MSECriterion
• Speed up Adam
• Speed up static graph execution
• Support reading TFRecord files from HDFS
• Support reading raw binary files from HDFS
• Check input size in concat layer
• Add proper exception handling for CaffeLoader&Persister
• + 7 more

🐛 Bug Fix

• Fix getModel failed in DistriOptimizer when model parameters exceed 2.1G
• Fix core number is 0 where there's only one core in system
• Fix SparseJoinTable throw exception if input’s nElement changed.
• Fix some issues found when save bigdl model to tensorflow format file
• Fix return object type error of DLClassifier.transform in Python
• Fix graph generatebackward is lost in serialization
• Fix resizing tensor to empty tensor doesn’t work properly
• Fix Adapter layer does not support different batch size at runtime
• + 5 more

📦 Highlights

• New protobuf-based model storage format
• Support model quantization
• Support sparse tensor and model
• Easier and broader Tensorflow model load support
• More layers/operations
• Apache Spark 2.2 support

✨ New Features

• Models & Layers & Operations & Loss function
• + Support convlstm3D model
• + Support Variational Auto Encoder
• + Support Unet
• + Support PTB model
• + Add SpatialWithinChannelLRN layer
• + Add 3D-deconv layer
• + Add BifurcateSplitTable layer
• + 43 more

📦 Enhancement

• 1. Support model quantization. User can speed up model inference by quantize the model
• 2. Display bigdl model in Tensorboard
• 3. User can easily convert a sequential model to graph model by invoking new added toGraph method
• 4. Remove unnecessary contiguous check in 3D conv
• 5. Support global average pooling
• 6. Support regularizer in 3D convolution layer
• 7. Add regularizer for convlstmpeephole3d
• 8. Throw more meaningful messages in layers and criterions
• + 18 more

🐛 Bug Fix

• 1. Fix TemporalConv layer cannot return parameter table
• 2. Fix some bugs when loading dilated group convolution from caffe
• 3. Fix some bugs when loading caffe v1 layers
• 4. Fix a bug in TimeDistributed layer
• 5. Fix get incorrect execution time in recurrent layers
• 6. Fix inplace layer clear state bug
• 7. Fix incorrect training data sample count under some input
• 8. Remove label check in BytesToGreyImg
• + 4 more

✨ New feature

• A new BigDL document website online https://bigdl-project.github.io/, which replace the original BigDL wiki
• Added New Models & Layers
• + TreeLSTM and examples for sentiment analytics
• + convLSTM layer
• + 1D convolution layer
• + Mean Absolute Error (MAE) metrics
• + TimeDistributed Layer
• + VolumetricConvolution(3D convolution)
• + 27 more

📦 Enhancement

• 1. Optimization method API refactor
• 2. Allow user to load a Caffe model without pre-defining a BigDL model
• 3. Optimize Recurrent Layers performance
• 4. Refine the ML pipeline related API, and add more examples
• 5. Optimize JoinTable layer performance
• 6. Allow user to use nio blockmanager on Spark 1.5
• 7. Refine layer parameter initialization algorithm API
• 8. Refine Sample class to save memory usage when cache train/test dataset as tensor format
• + 9 more

🐛 Bug Fix

• 1. Fix some error in Text-classifier document
• 2. Fix a bug when call JoinTable after clearState()
• 3. Fix a bug in Concat layer when the dimension concatenated along is larger than 2
• 4. Fix a bug in MapTable layer
• 5. Fix some multi-thread error not catch issue
• 6. Fix maven artifact dependency issue
• 7. Fix model save method won’t close the stream issue
• 8. Fix a bug in BCECriterion
• + 2 more

📋 Changes

• API Change
• Enhancement
• Bug Fix