LIBFMSR  0.1
Defines | Functions
fmsr.c File Reference

Implements core libfmsr functions callable by third-party apps. More...

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "fmsr.h"
#include "fmsrutil.h"
#include "matrix.h"
#include "misc.h"

Defines

#define NUM_CHECKS_THRESHOLD   1000000000
 Number of rounds to check in the two-phase checking during repair before declaring failure.
#define LAZY_THRESHOLD   512
 Number of rounds to try regenerating new chunks by heuristic coefficient generation instead of fully random coefficients.
#define CHECK_ERMDS   1
 Check erMDS property? 1 (yes) or 0 (no)
#define CHECK_MDS   0
 Check MDS property? 1 (yes) or 0 (no)
#define CHECK_RMDS   0
 Check rMDS property? 1 (yes) or 0 (no); Note: If yes, CHECK_MDS must be 1 as well.
#define NUM_T   7
 Number of threads to distribute matrix_mul() on.

Functions

void fmsr_init ()
 Must be called first before using any core functions in the library.
gf fmsr_nodeid (gf k, gf n, gf index)
 Returns id of the node where the index-th chunk resides, or 255 on failure.
gf fmsr_chunks_per_node (gf k, gf n)
 Returns the number of chunks per node, or 255 on failure.
gf fmsr_chunks_on_node (gf k, gf n, gf node, gf *chunk_indices)
 Returns the indices of chunks on a certain node through chunk_indices, or 255 on failure.
gf fmsr_nn (gf k, gf n)
 Returns the number of native chunks.
gf fmsr_nc (gf k, gf n)
 Returns the number of code chunks.
size_t fmsr_padded_size (gf k, gf n, size_t size)
 Returns the size of input file after being padded.
int fmsr_encode (gf k, gf n, gf *data, size_t data_size, int create_new, gf *code_chunks, gf *encode_matrix)
 Splits file into native chunks and encodes into code chunks.
int fmsr_decode (gf k, gf n, gf *code_chunks, size_t chunk_size, gf *chunk_indices, gf num_chunks, gf *encode_matrix, gf *decode_matrix, int create_new, gf *data, size_t *data_size)
 Decodes code chunks to give the original data.
int fmsr_repair (gf k, gf n, gf *encode_matrix, gf *erasures, gf num_erasures, fmsr_repair_hints *hints, gf *new_encode_matrix, gf *repair_matrix, gf *chunks_to_retrieve, gf *num_chunks_to_retrieve)
 Informs caller of the chunks and encoding matrix used in repair.
void fmsr_regenerate (gf *repair_matrix, gf rows, gf cols, gf *retrieved_chunks, size_t chunk_size, gf *new_code_chunks)
 Generates new code_chunks from existing code_chunks.

Detailed Description

Implements core libfmsr functions callable by third-party apps.

Author:
Henry Chen (chchen@cse.cuhk.edu.hk)

Define Documentation

#define CHECK_ERMDS   1

Check erMDS property? 1 (yes) or 0 (no)

#define CHECK_MDS   0

Check MDS property? 1 (yes) or 0 (no)

#define CHECK_RMDS   0

Check rMDS property? 1 (yes) or 0 (no); Note: If yes, CHECK_MDS must be 1 as well.

#define LAZY_THRESHOLD   512

Number of rounds to try regenerating new chunks by heuristic coefficient generation instead of fully random coefficients.

#define NUM_CHECKS_THRESHOLD   1000000000

Number of rounds to check in the two-phase checking during repair before declaring failure.

#define NUM_T   7

Number of threads to distribute matrix_mul() on.


Function Documentation

gf fmsr_chunks_on_node ( gf  k,
gf  n,
gf  node,
gf chunk_indices 
)

Returns the indices of chunks on a certain node through chunk_indices, or 255 on failure.

gf fmsr_chunks_per_node ( gf  k,
gf  n 
)

Returns the number of chunks per node, or 255 on failure.

int fmsr_decode ( gf  k,
gf  n,
gf code_chunks,
size_t  chunk_size,
gf chunk_indices,
gf  num_chunks,
gf encode_matrix,
gf decode_matrix,
int  create_new,
gf data,
size_t *  data_size 
)

Decodes code chunks to give the original data.

Parameters:
[in]k,n(n,k)-FMSR
[in]code_chunksretrieved code chunks
[in]chunk_sizesize of each code chunk
[in]chunk_indicesindices of retrieved chunks
[in]num_chunksnumber of chunks retrieved
[in]encode_matrixoriginal encoding matrix (can be NULL if decode_matrix is already supplied)
[in,out]decode_matrixdecoding matrix (can be NULL if not interested)
[in]create_newif 0, use supplied decode_matrix; else calculate and update decode_matrix
[out]datadecoded data (concat native chunks and unpad)
[out]data_sizesize of decoded data
Returns:
0 on success; -1 on failure
int fmsr_encode ( gf  k,
gf  n,
gf data,
size_t  data_size,
int  create_new,
gf code_chunks,
gf encode_matrix 
)

Splits file into native chunks and encodes into code chunks.

Note: sufficient memory should be allocated for the padded data (using fmsr_padded_size()), which is longer than data_size.

Parameters:
[in]k,n(n,k)-FMSR
[in]dataoriginal unpadded file to be encoded
[in]data_sizeunpadded file size
[in]create_newif 0, use supplied encode_matrix; else generate and update encode_matrix
[out]code_chunkscode chunks from encoded data
[in,out]encode_matrixencoding matrix
Returns:
0 on success; -1 on failure
void fmsr_init ( void  )

Must be called first before using any core functions in the library.

gf fmsr_nc ( gf  k,
gf  n 
)

Returns the number of code chunks.

gf fmsr_nn ( gf  k,
gf  n 
)

Returns the number of native chunks.

gf fmsr_nodeid ( gf  k,
gf  n,
gf  index 
)

Returns id of the node where the index-th chunk resides, or 255 on failure.

size_t fmsr_padded_size ( gf  k,
gf  n,
size_t  size 
)

Returns the size of input file after being padded.

void fmsr_regenerate ( gf repair_matrix,
gf  rows,
gf  cols,
gf retrieved_chunks,
size_t  chunk_size,
gf new_code_chunks 
)

Generates new code_chunks from existing code_chunks.

This is simply a single matrix multiplication that should be called after fmsr_repair().

Parameters:
[in]repair_matrixrepair matrix obtained from fmsr_repair()
[in]rows,colsnumber of rows and columns in repair_matrix
[in]retrieved_chunksall code chunks retrieved for the repair
[in]chunk_sizesize of each code chunk
[out]new_code_chunksall code chunks newly generated by this function
int fmsr_repair ( gf  k,
gf  n,
gf encode_matrix,
gf erasures,
gf  num_erasures,
fmsr_repair_hints hints,
gf new_encode_matrix,
gf repair_matrix,
gf chunks_to_retrieve,
gf num_chunks_to_retrieve 
)

Informs caller of the chunks and encoding matrix used in repair.

Most of the work during repair is done here.

Parameters:
[in]k,n(n,k)-FMSR
[in]encode_matrixcurrent encoding matrix before repair
[in]erasureslist of failed node indices
[in]num_erasuresnumber of failed nodes
[in]hintsrefer to struct fmsr_repair_hints
[out]new_encode_matrixnew encoding matrix after repair
[out]repair_matrixrepair matrix (follows order given by chunk list below)
[out]chunks_to_retrievelist of code chunks to retrieve from surviving nodes
[out]num_chunks_to_retrievenumber of code chunks to retrieve
Returns:
number of rounds checked on success (>0); -1 on unsupported parameters;
0 on failing to generate suitable coefficients
 All Data Structures Files Functions Variables Typedefs Defines