TreeBuilding documentation

class logflow.treebuilding.Dataset.Dataset(path_model='', name_model='', index_line_max=inf, path_data='', window_size=30, parser_function='')

A dataset is an object containing the data. It is used to load the files and to compute the window for each new prediction on a log.

Parameters:

• path_model (str, optional) – path to the model to load. Defaults to “”.
• name_model (str, optional) – name of the model to load. Defaults to “”.
• index_line_max (int, optional) – load only the lines with a lower index in the file. Avoid to load all the lines. Defaults to float(“+inf”).
• path_data (str, optional) – path to the logs. Defaults to “”.
• window_size (int, optional) – size of the window. Defaults to 30.
• parser_function (function, optional) – Function to split the log entry and get the message part. Defaults to “”, means split according to space and uses the words after the 9th position.

get_slice(index_line=-1)

Get the window associated with the line at the index_line.

Parameters:index_line (int, optional) – index of the line. Defaults to -1. Returns:return the log object with the slice added. If an error occurs, return -1. Return type:Log

load_files()

Load the files including word2vec, LSTM, counter and patterns.

load_logs()

Load the selected logs file up to the index_line_max.

show_selected_lines(index_line: int, range_line=100)

Show the selected and the range_line previous lines

Parameters:

• index_line (int) – the index of the selected line
• range_line (int, optional) – number of previous lines to print. Defaults to 100.

class logflow.treebuilding.Inference.Inference(models: List[T])

Manages the deep learning model, and run the inference through it.

Parameters:models (List) – list of the learned model to load.

probability(x: List[float]) → List[float]

Compute probability (ie 0 =< proba =< 1) values for each sets of scores in x.

Parameters:x (List[float]) – list of values. Here, it is used at the output of the attention layer Returns:list of probabilities Return type:List[float]

test(data: List[List[float]], log: logflow.treebuilding.Log.Log) → List[T]

Run the inference through the model and return only the value greater than the threshold.

Parameters:

• data (List[List[float]]) – vector to be used as an input
• log (Log) – log to predict

Returns:

list of the log with a weigth greater than the threshold.

Return type:

List

class logflow.treebuilding.Log.Log(line: str, index_line=-1, parser_function='')

Represents a line of log.

Parameters:

• line (str) – the line
• index_line (int, optional) – index of the line in the file. Defaults to -1.

preprocess_line()

Split the message part of the line

class logflow.treebuilding.Parser.Parser(dict_patterns, w2v, counter_patterns)

Get the pattern and the embedding of a log.

Parameters:

• dict_patterns (dict) – dict of patterns
• w2v (dict) – word2vec model
• counter_patterns (dict) – dict of the cardinality of patterns.

get_pattern(log: logflow.treebuilding.Log.Log)

Get the pattern associated with a log. Uses the method of Journal from the logsparser.

Parameters:log (Log) – log to be associated with a pattern .

get_w2v(log: logflow.treebuilding.Log.Log)

Get the embedding associated with a log and its pattern.

Parameters:log (Log) – log to be associated with the embedding.

class logflow.treebuilding.Tree.Tree

Manages the tree building according to the relation found by the Inference with the weigths of the attention value.

add_node(log: logflow.treebuilding.Log.Log, id=0, parent=-1, processed=False, weight=0)

Add a node to the tree

Parameters:

• log (Log) – log to add.
• id (int, optional) – id of the node. Defaults to 0.
• parent (int, optional) – id of the node’s parent. Defaults to -1.
• processed (bool, optional) – node has been merged or not. Defaults to False.
• weight (int, optional) – weigth associated to the node according to the attention layer. Defaults to 0.

get_node(index: int) → logflow.treebuilding.Node.Node

Get a node at the index

Parameters:index (int) – index of the node Returns:node at the index “index” Return type:Node

get_number_nodes_merged() → int

Return the number of merged nodes

Returns:number of merged nodes. Return type:int

merge_link()

Merge the links

merge_tree()

Merge the tree to keep only the relevant relations.

update_node(index: int)

Update the processed statut of a node at the index “index”

Parameters:index (int) – index of the node

class logflow.treebuilding.Node.Node(log: logflow.treebuilding.Log.Log, id: int, parent: int, processed: bool, weight: float)

Node of the tree

Parameters:

• log (Log) – Log associated to the node
• id (int) – id of the node
• parent (int) – id of the node’s parents
• processed (bool) – node has been merged or not
• weight (float) – weigth associated to the node according to the attention layer.

class logflow.treebuilding.Workflow.Workflow(dataset: logflow.treebuilding.Dataset.Dataset)

Computes the tree of correlations.

Parameters:dataset (Dataset) – dataset containing the data for the inference and the tree building.

detect_workflow(index_line: int) → str

Detect the workflow (i.e. the correlations tree)

Parameters:index_line (int) – index of the line Returns:representation of the three to be used with graphviz. Return type:str

get_tree(index_line: int)

Get the tree associated to the line at the index “index_line”

Parameters:index_line (int) – index of the line Returns:Index of the line is after the maximum loaded index. Return type:Exception

write_workflow_merged(local_tree: logflow.treebuilding.Tree.Tree) → str

Write the tree following the graphviz syntax

Parameters:local_tree (Tree) – tree to be written. Returns:representation of the tree. Return type:str