Theme of project

This project is an exemplary peer to peer, collaborative event, whereby people mutually form a search engine without the intervention of central servers or a central actor.

Motiviation

The direct motivation for this project is the increasingly worrying censorship and manipulation by major multinational companies of search engines (google, yahoo, microsoft and the censorship of the Chinese versions of their sites.) See also WhyOpenSearch.

Resulting product

The final product will be an easy to install multi-platform (MS Windows, Linux, Macintosh) client, which will not overload the computer. The client has a twofold goal: "crawling" the web for indexation and storage of the indexed data. Via the client the user may introduce search queries to open search, with will then be answered by the network.

The key-points are:

• Decentralisation, to prevent manipulation (censorship, biasing certain sites)
• Redundancy, to exclude single-points-of-failure and to make manipulation difficult
• Easily accessible: to maximize participation the number of possible technical obstacles will be kept to a strict minimum, so no complicated installation procedures and no excessive memory usage.

An important side product is a community of interested people, developers but also enthusiastic users (the early adopters). This community is necessary to keep on developing and distributing the product.

Target group

The target group is the internet user in the largest possible sense. The more internet users participate, the more indexed data will be gathered.

In order to reach the target group we will keep the software to be installed as easy as possible. Obstacles which might deter people must be avoided. Moreover, the client will be published in forums, conventions and meetings, via articles in magazines and of course by referring to it through our network.

A special category of internet user is the hacker/idealist/programmer who wants to help implement the project. This subcategory will be approached separately through our networks within these groups and by visiting meetings in the Netherlands and probably Belgium and Germany. Besides, we plan to organise three workshops (with virtual presence possible) to get this group involved.

Similar initiatives

Comparable projects do exist (see ExternalLinks). The difference with our project however is that the others do distributed crawling, but then use central storage for their results. It is explicitly our goal to exclude central storage.

Planning and results

Our initial schedule, submitted to digital pioneers (see also section Other parties, funding below) proposed the schedule below:

As of may 2007, it has become clear that this planning was unrealistic. A revised planning has been made, where we split tasks between the foundation and the developer. We also add two new actors: community and students. Building the community should result in work being split off to the community at some point. Also, the idea has been raised to work with students of different disciplines and integrate part of the work in their course material. With the increasing tendency of universities organising their curriculum around real-world projects, this might provide a good opportunity.

Software used

The open source/ free C Compiler gcc is used on unix/macos, on windows the gnu environment cygwin is used. For windows we will make use of an open source unix-socket-compatibility library. The user interface is entirely html/xml based, and is for access through a web browser exclusively. This eliminates the need to implement cross-platform compatible GUI code.

Content

Content is not the core of the project, but has an important auxiliary role. Good communication is vital to develop a community of developers (and probably later of users). To this end we will work via wiki (www.open-search.net), mailing lists and an IRC chat channel (most likely freenode.net).

Deliverables

The application will be available for free for the most frequently used versions of the supported operating systems (mac, windows, unix). During the project and thereafter the source codes will be available for free via a publicly accessible source code repository and be covered by the GPL (version 2) licence. All promotion material will be distributed with a creative commons licence. As is usual with free software projects, in the end it is the community which will continue the project through further development and support.

Organisation

The application is introduced by the govcom.org foundation ; for the implementation of the project itself a separate OpenSearchFoundation will be set up. The govcom.org foundation is an Amsterdam-based non-profit organisation; its aim is the development and hosting of political tools on the web. Examples are the issue crawler, which is software used to map debates on the web, a visualisation of the digital divide, and projects on internet censorship. govcom.org is financed by universities, research centers and social organisations. Projects are carried out by volunteers, paid professionals or by both.

The board of governors of the OpenSearchFoundation will be composed of:

• President: Dr. Richard Rogers, lecturer at Amsterdam University, founder of Govcom.org foundation (www.govcom.org), winner of the 2005 Best Information Science Book of the Year Award of the American society for Information Science and Technology (ASIST);
• Treasurer: Koen Martens, owner of Sonologic, programmer, hacker, organiser, treasurer of GroenLinks The Hague section and idealist;
• Secretary: initiator MSc Erik Borra, expert in the field of information technique and public administration, learning systems and text analysis.

The aim of the foundation is defined as follows:

The aim of the open search foundation is the development, distribution and maintenance of distributed, peer-to-peer search engines without central actors or storage. On the one hand this should prevent major multinationals to become preponderant in the field of information on the internet and on the other had to prevent public authorities to use the information concerning people's search behaviour to persecute them on the basis of their political convictions, sexual preferences or other characteristics.

As a developer we propose to hire Robin Gareus, MSc physicist who e.a. already developed a high performance network file system for the CERN cyclotron in Switzerland and is known here as an all round hacker and skilled programmer. He has already shown his interest in the assignment. Moreover, Erik Borra and Koen Martens will be responsible for the organisation, coordination, networks and public relations.

Budget

Labour costs for the developer include social insurance premiums, taxes and all legally required contributions .

Planning is as follows:

• December 2006: 0.8 fte for the development of the information architecture and design.
• January through March 2007: 0.8 fte for the development of the beta-version.
• April through the middle of June 2007: 0.4 fte for beta-testing and bug-fixing

Workshops

Three workshops will be organised to increase the involvement of potential developers. These workshops are organised around three special milestones: kick off (12 December 2006), the completion of the architecture and the start of the development (18 January 2007) and the start of the beta fase (19 April 2007). Renowned speakers will be invited to give a short presentation about that particular phase of the project to attract participants. There will be time available to brainstorm about the subject with an expert. Each workshop will be virtually accessible via a chat channel, projected on a screen, and a live audio and video stream. The workshops have two goals: to assemble interested people and to gather useful input. We count on approximately 20 participants.

Publicity

To attract a large public for the project, a number of publicity channels must be used. A friend designer will develop our foundation style and logo, which explains the low cost. As initiator Koen Martens owns a hosting company, websites and mailing lists can be developed for free and managed at a low cost.

To make the project known, reach potential developers and build a community, it is essential to participate in leading conferences in the field. Participation to the Chaos Communication Congress is intended: this is a yearly event organised by the chaos computer club. This meeting is attended by hackers from all over Europe and beyond, who are not only nerds, but also idealists, activists and people involved in the combination politics, society and ICT. The organisation reacted positively to our request to organise a workshop. This is an excellent opportunity to expand a community.

In the Netherlands and Belgium as well internet censorship and entanglement of interests in information flows get ever more attention; several organisations hold public meetings (such as HCC, nluug, de Balie, Creative Commons, Kennisland, political parties, education institutions). We will try to attend these events, if possible with an active contribution.

Other parties, funding

Initial funding came from the digital pioneers project, an initiative of the Netherlands Knowledgeland foundation to sparkle socialy committed innovative IT projects.

We are currently looking for further funding, candidates can be on ProjectFunding

Follow-up

The main feature of the project is that it is not meant to generate profits, so a commercial sequel is not intended. At the end of the first version the foundation will continue to exist, and if necessary look for subsequent financial sponsoring. It is important to make the community enthusiastic, so that volunteers from all over the world will contribute to its development according to the open source/free software model. Experience has shown that a living and active product is vital to reach this goal. Although our aim is not commercial, we are aware of the fact that this project is rather innovative. Enterprises and public authorities become interested as well in the possibilities of open source and peer-to-peer software. Of course this project will generate knowledge which could be interesting and useful for those parties.