As we ring in 2011, everyone is talking about Big Data.

The challenges surrounding Big Data — how to define it, how to manage and store it, how to make use of it — have commanded the attention of big industry players and start-ups alike, spawning cool new companies and open source projects

In the midst of all this activity, there is one growing class of data that has a particularly unique set of characteristics and database requirements: Machine-generated data. 

Machine-generated data is being defined in different ways by different people....

New! Infobright Enterprise and Community Editions 3.5 now GA

We're pleased to announce the GA versions of both Infobright Enterprise Edition (IEE) and Infobright Community Edition (ICE) 3.5. These new releases feature significant performance enhancements, as well as some other new features that users have asked for. We are not quite finished with all the performance testing, so additional information will be available later this month.

ICE and IEE Query Performance Improvements

Tests performed by Infobright have demonstrated the following performance gains in both IEE and ICE.

Other IEE and ICE Features

Also delivered in both ICE and IEE is the ability to load data from a remote machine, which allows the offloading of potentially heavy ETL processing to a separate server. This saves significant time for network transfer of large LOAD files, which can be a key limiter of load speed. In addition, both ICE and IEE may optionally use the MySQL query cache, resulting in significant performance gains for repeated execution of queries.

New to IEE

High Availability Replication
Using MySQL statement-based replication, this feature provides support for scale-out, spreading the load among multiple slaves to improve concurrency and performance. It includes support for both master/slave, master/multi-slave configurations, delivering a simple approach to high-availability (HA) and redundancy. It also improves data security and backup, allowing the replication process to be paused for backup services to be run on the slave.

Enhanced Workload Management
Building on the multi-core query processing delivering in IEE 3.4, new workload management features include parallel scanning and execution of individual queries and automatic query throttling technology for load balancing data loads and query execution.

Improved Database Monitoring
This improves management by monitoring database operations and resource usage including per-second CPU usage, physical memory usage, disk I/O, cache directory size, and query concurrency.

Supported platforms:

Case Study: Analyzing Network Traffic Faster with Open Source at Nokia Siemen

Register for this webinar to learn how Nokia Siemens Networks stores and analyzes terabytes of network data faster and at lower cost, using open source Hadoop with Infobright’s columnar database.

Some say it is strictly data that is generated without any direct human intervention, and others say it is data that includes the machine tracking of human activities as well, such as web log data. But whether the definition of machine-generated data is precise or a little more open, certain key characteristics nearly always apply:  new records are added with a high frequency, and the data itself is seldom if ever changed. Examples beyond web logs include all other types of logs, such as computer, network and security logs. Other examples include data coming from sensors. Finally, machine-generated data can also encompass call detail records, financial trading data, ATM transactions or RFID tags from shipping containers or cars on a toll way.