Intel® ISA-L Erasure Coding Sample Application Overview | Intel Software

Intel® ISA-L Erasure Coding Sample Application Overview | Intel Software


Hi. I’ve Praveen from Intel. In this video,
I’ll be walking you through the code sample portion
of the Intel Intelligent Storage Acceleration
Library [INAUDIBLE] coding example on the
Intel Developer Zone. You can find a link to this
example in the description. Reed-Solomon error
correction schemes are well-known and have
been in use for long time. In this example, the Intel ISA-L
implementation of Reed-Solomon will be used to
demonstrate that it is now possible to keep up with
the current storage hardware throughput requirements. It can be done by doing a RS
error correction in software instead of offloading
to dedicated hardware or using [INAUDIBLE]
parity computations. In other words, using Intel’s
ISA-L RS implementation, error correction can
be performed on the CPU sufficiently fast enough
to not induce a bottleneck or add substantial latency
to storage devices. The sample
application creates up to 24 different buffers
in memory to store data. A larger data set of up
to 256 megabytes of data is distributed
across these buffers. ISA-L’s RS algorithm
implementation is used to calculate error
correction codes that are stored and distributed with
the data across the buffers. There are two things to consider
here, storage time and recovery time. Storage time is the time taken
to generate error correction codes and storing them. And recovery time is the time
taken to recover the lost data from the error correction code. To simulate failure, up
to two memory buffers are made inaccessible. And the program
demonstrates that the data can be completely and
correctly recovered from the remaining buffers. A recovery time is also
measured and logged. In this example,
the program will run using 24 buffers in
total with a total data size of 256 megabytes. And two of those
buffers will be lost. Therefore, two buffers also used
to hold error correction codes. And the other 22 are
holding the data. From the output,
the program displays the results of the
storage and recovery times applying the Intel ISA-L
Reed-Solomon method. The results illustrate that with
the Intel [INAUDIBLE] process server, the
Reed-Solomon computation can be done efficiently and
quickly, as demonstrated in the example runs. There are three key points
we can take from this. One, it shows the flexibility
of Reed-Solomon implementation. You can select any number
of parity and data chunks to split things. And it’s very flexible. The second point is that we are
measuring the latency numbers. And we are measuring time. If we work back
these numbers, we get to the throughput numbers. And that is what I
called out previously. If you do 256 megabytes
in 46 milliseconds, or 4,600 microseconds,
you work backwards. And you get the 5.5 gigabytes
per second throughput. The third point
is this optimizes the latency and throughput. And as it is configurable
and flexible, you can deploy it in
number of servers, or if not servers, nodes
within your individual server. The example shows a method on
how to incorporate the Intel ISA-L Reed-Solomon
error correction feature into your
storage applications and shows how to be
able to prepare the data and recover data, which
can help you on how to adapt to the technology. Thanks for watching. To learn more about ISA-L,
check out the links below. And make sure to watch the
rest of this video playlist on Intel ISA-L. Don’t forget to
like this video and subscribe.

Leave a Reply

Your email address will not be published. Required fields are marked *