+ Reply to Thread
Results 1 to 7 of 7
  1. #1

    Distributed Load Processor (DLP)

    Hi,

    I wanted to learn little bit more about Distributed Load Processor (DLP).
    I read the material and wanted to make sure I understood all the concept.
    Is it correct to say that DLP can run on any device - local or remote - that runs specific Windows or Unix OS even if that device does not run Infobright engine?
    For example, would this work:

    Server S is running IEE and workstations w1, w2 w3, ... wN are distributed (remote) machines that do not run Infobright database.
    Can w1, w2, w3, ... wN send per-processed and compressed data packs over WiFi or other network?

    In addition, there is a statement in the User Guide:
    ... You can use the Distributed Load Processor to convert data from binary format to the INFOBRIGHT data format.
    I'm not sure I understand it, what kind of binary input this statement is referring to? and by Inforbright data format do you mean loading to the database?

    C_G

  2. #2

    Distributed Load Processor (DLP)

    Hi C_G,

    You're correct on all statements.

    DLP allows you to offload the load process to remote servers. The output of these processes is a heavily compressed, knowledge-grid created output. When sent to the database, it's almost as simple as tacking it on to the end of the DB. The overall impact on the database server is very minimal. In IEE, you can achieve about 2TB/hr into a single table using this methodology.

    As for binary, we mean the binary export of Infobright. For example, set @bh_dataformat='binary' and export data. That binary data can be consumed by DLP.

    Cheers,

    Jeff

  3. #3

    Distributed Load Processor (DLP)

    Thanks Jeff,

    I also wanted to ask about the type of connection. Can DLP work over Internet?
    My use case is involving field operators that are taking measurements inf field where sometimes there is a connection and some times not.
    Can DLP per-process or do some work with data while outside the cell coverage and resume loading once the connection is re-established?
    Thanks
    C_G

  4. #4

    Distributed Load Processor (DLP)

    DLP Can work over a TCP/IP connection, but the Keep-Alive settings are not exposed. Depending upon what stage of the process, i.e. (validating schema, extracting data, processing data, creating data file, loading data files) the connection is pinged at intervals to maintain the tcp/ip connection without each of the endpoints closing, this is a standard process in many tcp/ip connection applications. There are a few factors involved, so even though it may work sometime becuase the connection was alive when it needed it in it's own process and was able to complete, I would not consider it a great way of ensuring accuracy and completion and existance of data when in changing environment like you are describing.

  5. #5

    Distributed Load Processor (DLP)

    Thanks Craig,

    This answers my question.
    C_G

  6. #6
    hi all,

    Im not getting DLP user guide in infobright forum can anyone upload it plz................

  7. #7
    The DLP User Guide is in the Support Portal. Rajaduraibanu, are you working with DLP Trial or IEE ?

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts