Our long-term mission since 2005 is to develop community-based Collective Mind infrastructure and public repository of knowledge for systematic, collaborative and reproducible research and experimentation in computer engineering based on practical needs for our R&D on auto-tuning, machine learning and crowdsourcing! If you are interested in Collective Mind's preview, tutorials, presentations, current industrial usages, or new publication models, please contact Grigori Fursin!

cTuning-related recent and upcoming events and news:

Home

Continuing innovation in science and technology is vital for our society and requires ever increasing computational resources. However, delivering such resources became intolerably complex, ad-hoc, costly and error prone due to an enormous number of available design and optimization choices combined with the complex interactions between all software and hardware components, and a large number of incompatible analysis and optimization tools. As a result, understanding and modeling of the overall relationship between end-user algorithms, applications, compiler optimizations, hardware designs, data sets and run-time behavior, essential to provide better solutions and computational resources, became simply infeasible as confirmed by many recent long-term international research visions about future computer systems. Additional problems are caused by the lack of a common experimental methodology, lack of interdisciplinary background, and lack of unified mechanisms for knowledge building and exchange apart from numerous similar publications where reproducibility and statistical meaningfulness of results as well as sharing of data and tools is often not even considered in contrast with other sciences including physics, biology and artificial intelligence. In fact, it is often impossible due to a lack of common and unified repositories, tools and data sets. At the same time, there is a vicious circle since initiatives to develop common tools and repositories to unify, systematize, share knowledge (data sets, tools, benchmarks, statistics, models) and make it widely available to the research and teaching community are practically not funded or rewarded academically where a number of publications often matter more than the reproducibility and statistical quality of the research results. As a consequence, students, scientists and engineers are forced to resort to some intuitive, non-systematic, non-rigorous and error-prone techniques combined with unnecessary repetition of multiple experiments using ad-hoc tools, benchmarks and data sets. Furthermore, we witness slowed down innovation, dramatic increase in development costs and time-to-market for the new embedded and HPC systems, enormous waste of expensive computing resources and energy, and diminishing attractiveness of computer engineering often seen as "hacking" rather than systematic science (detailed vision is available in these two open access publications 2009, 2013). Finally and thankfully, we started seeing similar initiatives at some major conferences including OOPSLA and PLDI!

Currently, we see 2 main paths for computer engineering:

  • continue developing various ad-hoc techniques and publish non-reproducible experimental results.
  • join long-term, collaborative initiative to make design, analysis and optimization of computer systems systematic and reproducible, and to change current publication model to favor reproducible and truly collaborative research [M1, Vision].

Since 1996, we are working on gradual systematization of knowledge about design and optimization of computer systems based on our background in physics and AI. 10 years later, we started preparing and releasing public and collaborative experimental methodology, repository, tools and publication model for computer engineering (cTuning / Collective Mind initiative) that favors collaborative discovery, sharing and reuse of knowledge [P1, M1]. This technology allows users to:

  • share experimental data, tools, models and interfaces (in workgroups through private cTuning repository or with anyone through public cTuning repository)
  • collaboratively explore large optimization spaces
  • apply classification and predictive models to existing data to explain complex behavior of existing computer systems
  • validate data and models by the community (similar to physics, biology and other sciences)
  • collaboratively identify anomalies in behavior and suggest how to improve it (statically or dynamically)
  • extrapolate this knowledge to build more efficient computer system in terms of performance, power, size, reliability and other characteristics

Having common collaborative R&D repository and infrastructure allows users to focus their effort on novel approaches combined with with data mining, classification and predictive modeling rather than spending considerable effort on building new tools with already existing functionality or using some ad-hoc tuning heuristics. It also allows conferences and journals to favor publications that can be collaboratively validated by the community.

Since 2007, we have released all experimental data and tools for our cTuning / Collective Mind-related publications [TR2013, TR2009, IJPP2011, ACM TACO2010, PLDI2010, SMART2009, HiPEAC 2009] which allowed further collaborative validation, reproducibility and extension of this technology together with IBM, CAPS, ARC, Intel/CEA Exascale Lab, Google, University of Edinburgh, ICT, UPC, NCAR!

We believe that cTuning may finally complete the puzzle about how to consolidate various ad-hoc techniques and tools together to build efficient self-tuning computer systems. Therefore we strongly advocate for further collaborative R&D and new publication model. If you are interested, join us at upcoming events or use our cTuning and Collective Mind mailing lists to participate in discussions and collaborative developments.

You can find more details about history and implementation of cTuning here.

We would like to thank all our colleagues for interesting and sometimes tough discussions, feedback, collaborations and support during development of cTuning technology.