Tasks Manual

1. Network
    <i>These tasks are useful to receive or send data over the network</i>

    Available Tasks:      
    • ftp_in
    • mqtt_in
    • seedlink_in
    • http_out
    • s3_out

2. Reader
   <i>These tasks are useful to read data from file</i>
    
    Available Tasks:      
    • csv_reader

3. Data Transformation
<i>Each of these tasks transforms data in some way</i>

•	Aggregator: Combines data from multiple sources.
•	Classifier: Assigns categories or labels to data based on attributes.
•	Mapper: Maps data fields from one schema to another.
•	Converter: Converts data formats or types.
        
Available Tasks:      
• id_mapper  
• payload_converter
• timeslot_aggregator

5. Machine_learning
<i> These task perform machine learning model to extract information and insight from data </i>

 Available Tasks:
 • water_leak_detection

•	broker_url: Address of the MQTT broker to connect to.
•	broker_port: Port of the MQTT broker. The default is 1883 for unencrypted connections.
•	mqtt_topics: List of topics to subscribe to. You can use wildcards like # for multiple subscriptions.
•	qos: Quality of Service level for message delivery. Can be 0, 1, or 2.
•	keep_alive: Keep-alive interval in seconds to maintain the connection to the broker.
•	use_tls: Boolean indicating whether to use an encrypted TLS connection. Set to true if the broker requires TLS.
•	username and password: Credentials for authentication with the MQTT broker, if required.
•	tls_params: Additional parameters for TLS configuration, such as certificates and certificate requirements.
•	Example of tls_params:

•	seedlink_server: Address and port of the SeedLink server to connect to (e.g., "hostname:port").
•	sensors: List of sensor codes from which to receive data. Each code corresponds to a specific sensor.
•	duration: Duration in seconds to keep the connection to the SeedLink server open. Set to 0 or omit for a continuous connection.
•	dependencies: As an input task, it generally has no dependencies.
•	instances: Number of instances of the task. Usually 1 to avoid conflicts in data reception.

•	api_key: API key used for authentication with the HTTP endpoint. Replace "changeMe" with your actual API key.
•	endpoint: URL of the endpoint to send data to via HTTP POST.
•	compress: Boolean indicating whether to compress the data before sending. true to enable compression (typically gzip), false otherwise.
•	batch_size: Number of messages to accumulate before sending them in a single batch. A value of 1 means data is sent immediately without batching.
•	dependencies: Depends on the previous task, such as json_to_payload_converter.
•	instances: Number of parallel instances of the task. Increasing this value may improve sending speed in high-volume scenarios.

•	csv_basepath: Path to the directory containing the CSV files to process. The task will read files present in this directory.
•	processing_dir: Directory where CSV files are moved during processing. This helps prevent re-reading the same files.
•	processed_dir: Directory where CSV files are moved after complete processing.
•	delimiter: The delimiter used in the CSV files (e.g., , for standard CSV or ; for semicolon-separated CSV).
•	header: Indicates whether the CSV file has a header row. 0 means the first row is the header. Set to None if there is no header.
•	use_in_memory: Boolean that determines whether the CSV file should be read entirely into memory (true) or streamed (false). Streaming is recommended for very large files.
•	chunksize: Number of rows to read per chunk during in-memory processing. Helps manage memory usage when reading large files.
•	dependencies: List of tasks that this task depends on. For csv_reader, this is usually empty because it is an input task.
•	instances: Number of parallel instances of this task to run. Usually 1 to avoid conflicts in file reading.

•	mapping: A dictionary that defines how fields in the input JSON should be mapped in the output payload. The key is the original field name, and the value is the desired field name in the payload.
•	For example, "CA1": "00.CA.acceleration_x" means that the CA1 field in the input JSON will be mapped to 00.CA.acceleration_x in the output payload.
•	dependencies: List of tasks that this task depends on. In this case, it depends on "task1", which could be an input task like csv_reader or seedlink_in.
•	instances: Number of parallel instances of the task. Can be increased to improve performance if necessary.

•	Instances: Increasing the number of instances for a task can improve performance through parallelism but may also increase resource management complexity. Use this parameter judiciously.