Phase-change memory models for deep learning training and inference

S. R. Nandakumar; Irem Boybat; Vinay Joshi; Christophe Piveteau; Manuel Le Gallo; Bipin Rajendran; Abu Sebastian; Evangelos Eleftheriou

doi:10.1109/ICECS46596.2019.8964852

Phase-change memory models for deep learning training and inference

S. R. Nandakumar, Irem Boybat, Vinay Joshi, Christophe Piveteau, Manuel Le Gallo, Bipin Rajendran, Abu Sebastian, Evangelos Eleftheriou

Research output: Contribution to journal › Conference paper › peer-review

14 Citations (Scopus)

Abstract

Non-volatile analog memory devices such as phase-change memory (PCM) enable designing dedicated connectivity matrices for the hardware implementation of deep neural networks (DNN). In this in-memory computing approach, the analog conductance states of the memory device can be gradually updated to train DNNs on-chip or software trained connection strengths may be programmed one-time to the devices to create efficient inference engines. Reliable and computationally simple models that capture the non-ideal programming and temporal evolution of the devices are needed for evaluating the training and inference performance of the deep learning hardware based on in-memory computing. In this paper, we present statistically accurate models for PCM, based on the characterization of more than 10, 000 devices, that capture the state-dependent nature and variability of the conductance update, conductance drift, and read noise. Integrating the computationally simple device models with deep learning frameworks such as TensorFlow enables us to realistically evaluate training and inference performance of the PCM array based hardware implementations of DNNs.

Original language	English
Pages (from-to)	727-730
Number of pages	4
Journal	2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019
DOIs	https://doi.org/10.1109/ICECS46596.2019.8964852
Publication status	Published - Nov 2019
Event	26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019 - Genoa, Italy Duration: 27 Nov 2019 → 29 Nov 2019

Keywords

Deep learning
Inference
Phase-change memory
Statistical model
Training

Access to Document

10.1109/ICECS46596.2019.8964852

Cite this

@article{818a7f1c495b46a5ab6437a28cbe9db3,

title = "Phase-change memory models for deep learning training and inference",

abstract = "Non-volatile analog memory devices such as phase-change memory (PCM) enable designing dedicated connectivity matrices for the hardware implementation of deep neural networks (DNN). In this in-memory computing approach, the analog conductance states of the memory device can be gradually updated to train DNNs on-chip or software trained connection strengths may be programmed one-time to the devices to create efficient inference engines. Reliable and computationally simple models that capture the non-ideal programming and temporal evolution of the devices are needed for evaluating the training and inference performance of the deep learning hardware based on in-memory computing. In this paper, we present statistically accurate models for PCM, based on the characterization of more than 10, 000 devices, that capture the state-dependent nature and variability of the conductance update, conductance drift, and read noise. Integrating the computationally simple device models with deep learning frameworks such as TensorFlow enables us to realistically evaluate training and inference performance of the PCM array based hardware implementations of DNNs.",

keywords = "Deep learning, Inference, Phase-change memory, Statistical model, Training",

author = "Nandakumar, {S. R.} and Irem Boybat and Vinay Joshi and Christophe Piveteau and {Le Gallo}, Manuel and Bipin Rajendran and Abu Sebastian and Evangelos Eleftheriou",

year = "2019",

month = nov,

doi = "10.1109/ICECS46596.2019.8964852",

language = "English",

pages = "727--730",

journal = "2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019",

note = "26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019 ; Conference date: 27-11-2019 Through 29-11-2019",

}

TY - JOUR

T1 - Phase-change memory models for deep learning training and inference

AU - Nandakumar, S. R.

AU - Boybat, Irem

AU - Joshi, Vinay

AU - Piveteau, Christophe

AU - Le Gallo, Manuel

AU - Rajendran, Bipin

AU - Sebastian, Abu

AU - Eleftheriou, Evangelos

PY - 2019/11

Y1 - 2019/11

N2 - Non-volatile analog memory devices such as phase-change memory (PCM) enable designing dedicated connectivity matrices for the hardware implementation of deep neural networks (DNN). In this in-memory computing approach, the analog conductance states of the memory device can be gradually updated to train DNNs on-chip or software trained connection strengths may be programmed one-time to the devices to create efficient inference engines. Reliable and computationally simple models that capture the non-ideal programming and temporal evolution of the devices are needed for evaluating the training and inference performance of the deep learning hardware based on in-memory computing. In this paper, we present statistically accurate models for PCM, based on the characterization of more than 10, 000 devices, that capture the state-dependent nature and variability of the conductance update, conductance drift, and read noise. Integrating the computationally simple device models with deep learning frameworks such as TensorFlow enables us to realistically evaluate training and inference performance of the PCM array based hardware implementations of DNNs.

AB - Non-volatile analog memory devices such as phase-change memory (PCM) enable designing dedicated connectivity matrices for the hardware implementation of deep neural networks (DNN). In this in-memory computing approach, the analog conductance states of the memory device can be gradually updated to train DNNs on-chip or software trained connection strengths may be programmed one-time to the devices to create efficient inference engines. Reliable and computationally simple models that capture the non-ideal programming and temporal evolution of the devices are needed for evaluating the training and inference performance of the deep learning hardware based on in-memory computing. In this paper, we present statistically accurate models for PCM, based on the characterization of more than 10, 000 devices, that capture the state-dependent nature and variability of the conductance update, conductance drift, and read noise. Integrating the computationally simple device models with deep learning frameworks such as TensorFlow enables us to realistically evaluate training and inference performance of the PCM array based hardware implementations of DNNs.

KW - Deep learning

KW - Inference

KW - Phase-change memory

KW - Statistical model

KW - Training

UR - http://www.scopus.com/inward/record.url?scp=85075017281&partnerID=8YFLogxK

U2 - 10.1109/ICECS46596.2019.8964852

DO - 10.1109/ICECS46596.2019.8964852

M3 - Conference paper

SP - 727

EP - 730

JO - 2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019

JF - 2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019

T2 - 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019

Y2 - 27 November 2019 through 29 November 2019

ER -

Phase-change memory models for deep learning training and inference

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this