Competition rules (Draft 1.0)

Note: Significant changes to these rules will be announced on the ioi-announce mailing list as well as the newsgroup uw.ioi2010 (which can be accessed through many newsreaders as well as Google Groups).

Overview of changes

Here is an overview of changes to the competition rules from IOI 2009 to IOI 2010:

1. On each competition day contestants will be given four tasks to complete in the five hours. Each task will be subdivided into two or more subtasks. Subtasks are of graduated difficulty, from straightforward to more challenging.
2. Each subtask will have either a fixed score or open-ended score. Points will be awarded for fixed-score subtasks only if the submission passes all tests for the subtask. Points will be awarded for open-ended tasks according to a formula given in the task statement.
3. Feedback will be available to the contestant for a limited number of submissions per task. Using feedback, contestants will be able to determine with reasonable certainty whether or not each subtask has been solved.
4. During the competition a summary scoreboard will be available to spectators but not to contestants. The scoreboard is provided in accordance with a one-year exemption to the IOI regulation that prohibits the release of scores prior to the closing ceremony.
5. There will be two kinds of tasks:
   1. Programming tasks involve implementing one or more procedures that solve the task as described in the task statement.

      Three Demonstration tasks illustrate the contestant setup.

      The grader will do all input and output and call the submitted procedures. A prototype grader will be supplied for all tasks, which contestants may use or adapt for testing. The official grader will implement the same interface as the prototype available to competitors, but will employ additional security measures and, of course, different test examples. The official grader will run on a special grading machine, with identical hardware configuration.

      There is no specific distinction between the former batch and reactive tasks. As noted above, all programming tasks will be solved by procedures, and the grader will supply the mainline. Submissions may be required to interact with the grader by calling library procedures, which are implemented by the grader.

   2. Output-only tasks involve submitting one or more text files that are solutions to the task for specific inputs that are supplied to contestants.
6. A special interactive development environment (IDE) called RunC will be available. RunC allows contestants to build and execute their programs, to test them with various inputs, and to submit them to the grader. RunC may be invoked using a hotkey in the gedit editor, or from the command line.
7. All submissions will be graded as soon as reasonably possible. For each task, the results of one simple test will be available to contestants once grading is complete. Here is an illustrated tutorial on how to view results. Results of other tests may be viewed a limited number of times per task. These results are derived from the official grading data and may be used to infer provisional scores.

IOI 2010 Competition Rules

These Competition Rules cover Competition Procedures and Judging Procedures. This draft may be revised to address omissions or inconsistencies, but will not change substantially. The final version will be presented at the first GA meeting of IOI 2010.

Delegation Leaders have the responsibility of ensuring that all members of their delegation fully understand these rules and abide by them.

 

Tasks

All of the tasks in IOI 2010 are designed to be algorithmic in nature. All tasks will be divided into subtasks of graduated difficulty. Levels of difficulty may be distinguished by:

1. time and memory constraints,
2. optimality criteria,
3. simplifying assumptions,
4. other amendments to the task specification.

All submissions will be subject to general time and memory constraints. The CPU and memory limits will be at least double what is needed.

Contestants are advised to reason about whether a possible approach will meet all the constraints of a subtask. The grader will enforce all constraints.

Tasks may be of the following types:

1. Programming tasks : Solutions comprise one or more procedures implemented in one or more source files, which may be tested and submitted using the RunC programming environment. An interface will be given for each procedure to be implemented by the contestants, and for each procedure supplied by the grader. Skeleton implementations for all tasks will be available on contestant workstations. The grader may read input data and specifications from a file, but the submitted procedures do not access this file directly: all parameters and results are communicated between the grader and the submission by procedure parameters and results.

   Where the task requires the submission of more than one procedure, each must be implemented in a separate file, as specified in the task statement. The two procedures must not communicate other than through the specified interface. Shared variables, file access and network access are forbidden. The skeleton grader implementation given to contestants may not enforce these restrictions, but the official grader will. Any deliberate attempt to defeat the official grader's enforcement of the restrictions may subject the contestant to disqualification. On the other hand, contestants may modify the skeleton grader on their workstation as they see fit.

2. Output-only tasks : Solutions comprise a set of "output" data files. The files must be placed in a task-specific directory; the RunC programming environment is used to submit them all at once to the grader.

For every task, CPU and memory limits will be specified. The CPU limit will be specified in seconds, as measured by the operating system. The memory limit is on the overall memory usage including executable code size, a stack, a heap, etc. In general, these limits will be at least double those required by the expected solution.

Submissions must not do input and output, and must not use any system calls. Any contestant who attempts to compromise the system may be subject to disqualification.

Programming Tasks

The source program provided by the contestant must be contained in one or more source files as specified in the task statement. The task statement will define:

1. the task to be solved,
2. the interfaces to the procedures to be implemented and procedures implemented by the grader
3. the input value ranges (when applicable),
4. the resource limitations for the computations (e.g. CPU time, memory limitations),
5. for subtasks where partial scores are possible, the criteria by which these scores are calculated,
6. any other constraints on the program.

The contestant workstation will contain one folder for each programming task, which will contain the necessary interface files, a sample grading program, and a skeleton implementation of the required source files which exercises the interface but does not correctly solve the problem.

The submit command may be used within the folder to submit all required files for grading. Alternatively, the submit plugin of gedit may be invoked by typing Control-J (for judge).

Output-only tasks

The competition may include tasks for which input data is given to the contestant and the contestant is required to produce only the output data as a solution. If the contestant writes programs to help determine the output data, the programs are not graded with the solution. The input data will be provided in ASCII text files. For these tasks, the task statement will specify:

1. the task to be solved,
2. the structure of the input and output files,
3. the names of the input files,
4. the names of output files,
5. grading criteria.

The submit command may be used within the folder to submit all required files for grading. Alternatively, the submit plugin of gedit may be invoked by typing Control-J (for judge).

Competition Procedures

Competition Schedule

There will be two competition days. On each competition day contestants will be given four tasks to complete in the five hours. Every task will be divided into subtasks with graduated difficulty, and contestants will have access to detailed feedback for the entire set of the official test cases. The schedule will be published on the official competition web site.

There will be a two hour Practice Competition prior to the first competition day. The purpose of this practice round is to familiarize all contestants with the environment and procedures. The practice tasks will be distributed before the IOI. Contestants may bring printed solutions to the practice tasks, on paper only. All contestants are encouraged to take part in the Practice Competition round.

Quarantine

In order to protect the confidentiality of the tasks, all direct and indirect contacts and communication between contestants and delegation leaders are prohibited between the moment where tasks for a competition day are presented to the members of the GA and the end of the five-hour round on the following day. During this period the contestants are not allowed to communicate by any means, direct or indirect, with any member of the GA or anyone who has seen the tasks. The GA members are not allowed to communicate task-related information to anyone who is not a member of the GA. The contestants, the GA members and anyone else who has had access to the tasks must obey any instructions which restrict their access to specific parts of the IOI venue.

If a contestant violates the quarantine, he or she can be subject to disqualification. If some other person associated with a national delegation violates the quarantine, then all contestants of that delegation may be subject to disqualification.

In order to eliminate any suspicion of cheating, contestants and team leaders should not use mobile phones and should switch off all wireless facilities in all devices during the quarantine.

Competition Equipment and Environment

All contest machines will be installed to boot in Linux. The following packages will be installed:

1. Ubuntu 10.4 i386; GNOME;

2. web browsers: Firefox;

3. editors: mcedit, joe, vim, kate, kwrite, emacs, lazarus, gedit, nano, scite, codeblocks, geany, fp

4. compilers: gcc 4.4.3, g++ 4.4.3, fpc 2.4.0

5. debuggers: gdb, ddd, valgrind;

6. Custom RunC programming environment, with automatic run, test and submit commands and plugins. Support for C, C++, FPC.

7. Documentation for Pascal, C, C++, STL.

Hardware specification:

1. Intel Core i5-650 Processor (3.20GHz,4M Cache)

2. 3GB PC3-10600 1333MHz SoDIMM (1 DIMM)

3. 250GB, 8M Cache, 7200RPM SATA II

4. Standard keyboard (US layout)

In order to translate tasks, team leaders will be provided with computers with a version of Microsoft Word with some multi-language support. In addition, Ethernet ports will be available for leaders bringing laptops.

Competition Supplies

In the competition room, blank paper and writing utensils will be provided. On the competition days, contestants may not bring anything into the competition rooms, but:

1. clothing,
2. reasonable jewelry,
3. writing utensils,
4. simple wristwatches,
5. simple keyboards (see below),
6. small mascots (see below),
7. English dictionaries (see below).

If a contestant wants to bring a keyboard, small mascots or English dictionaries to the competition, these must be submitted to the technical staff during the practice competition day. Any of these will be checked and, if cleared, will be given to the contestant in the first competition day. After the first competition day, the contestant must leave these items on his or her workstation if he or she wants to use them during the second competition day. Once again these will be checked and, if cleared, will be given to the contestant in the second competition day. After the second competition day the contestant must take any of these items with him or her.

Any attempt to bring any other item into the competition room will be considered cheating. In particular, during competition rounds it is strictly prohibited to bring:

1. any computing equipment (e.g., keyboards, mice, calculators, organizers, PDAs, computers, calculator or "data bank" watches),
2. any books, manuals, written or printed materials,
3. any data storage medium (e.g., diskettes, CD-ROMs, "pen drives", compact flash/smart cards, micro-drives),
4. any communication devices (e.g., mobile phones, radios of any sort).

Any electronic or printed materials provided by the organizers during the Competition Round may be used by the contestants (e.g., a Users Guide to the Contest System, or any electronic documentation or reference manuals provided in the installed contest environment or on the provided Contest Server).

Starting the Competition

On the competition days, immediately after breakfast ends, contestants will be admitted to the competition hall. All contestants must wear their ID badges during the competition. Each contestant will have a pre-assigned workstation; this information will be posted in the morning of the competition day. Contestants should be in their seats by at least 5 minutes prior to the start of the competition . On the way to their computers contestants will be checked to verify that they are only bringing allowed items into the competition hall. Small mascots and dictionaries must be submitted to the technical staff during the practice competition day, check Competition supplies section. Contestants must find their assigned computer, sit down, and wait for the competition to begin without touching any:

1. computers,
2. keyboards,
3. mice,
4. other competition materials including tasks.

The beginning of the competition will be marked by an audible start signal, after which the contestant may use any of the above items.

Clarification Requests

During the first two hours and a half of competition, contestants may submit written questions concerning any ambiguities or items needing clarification in the competition tasks. After this period comments may be received, but no answers will be given. Questions and comments must be submitted on the provided Clarification Request Forms, expressed either in the contestant's native language or in English. If required, delegation leaders will translate their contestants' questions into English after they are submitted and before they are sent to the Scientific Committee. The Scientific Committee will respond to every question submitted by the contestants. Since this might take some time, contestants should continue working while waiting for the answer to their questions. Contestants should phrase their questions so that a yes/no answer will be meaningful. Questions will be answered with one of the following:

1. "YES"
2. "NO"
3. "ANSWERED IN TASK DESCRIPTION (EXPLICITLY OR IMPLICITLY)" - The task description contains sufficient information. The contestant should read it again carefully.
4. "INVALID QUESTION" - The question is most likely not phrased so that a yes/no answer would be meaningful. The contestant is encouraged to rephrase the question.
5. "NO COMMENT" - The contestant is asking for information that the Scientific Committee cannot give.

Contestants will not be involved in or exposed to discussion regarding their questions. However, in cases where the Scientific Committee feels that the contestant does not understand the task statement, English text which helps clarify the task may be added to the response, and if necessary, translated by the contestant's Delegation Leader into the language in which the question was originally posed.

Competition Server

The IOI competition takes place in a distributed environment in which each competitor has his or her own workstation for development and initial testing. Grading and evaluation take place on the Competition Server, which provides a similar execution environment to that of the contestant workstation. The hardware of the Competition Server is the same as that of the contestant's workstations.

The grading server uses RunC to grade programs in much the same way that contestants may use RunC to develop and test them on their workstations. However, the actual grading program and test examples are more rigorous.

Each contestant's workstation has only Linux installed. A set of software development tools will be available. These tools are used to develop solutions to the competition tasks. There is no restriction on the number of times a program may be edited, compiled, and run on the workstation.

The workstations will have network access to the Competition Server and facilities such as printing, test execution, and solution submission.

Facilities provided by the Competition Server are described next. These descriptions as well as the whole competition rules will be available on-line on the Competition Server.

The workstations will neither have access to each other nor the Internet. Any attempt to access another competitor's computer or the Internet will be considered cheating. Attempts to tamper with the Competition server, such as probing the server with customized URL's, will also be considered cheating.

Printing

Contestants are allowed to print only text files during the competition. After a contestant requests that a document be printed via the Contest Server interface, the support staff will deliver the printout to the contestant; contestants should not leave their computer to find printouts. Printouts will be delivered as quickly as possible, but large volumes can produce delays in delivery. The limit on the length of a printed document is 10 pages and 50 KB. If a document to be printed is longer, only the first 10 pages of the first 50 KB will be printed. On each competition day the contestant can print at most 30 documents.

SUBMIT Interface

Contestants must submit their solutions by using either the submit command or Control-J within the gedit editor. The appropriate task name and submission files are determined by the current directory (folder) when the submit command is executed. Participant workstations will be pre-configured with such a directory for each task.

The programming language used by the contestants is determined based on the solution file extension:

1. Pascal - *.pas,
2. C - *.c,
3. C++ - *.cpp,

Solutions for tasks may be submitted at any time. For tasks that require programs as solutions, each submitted source program must be written in C, C++ or Pascal, it must be smaller than 100 KB, the evaluation server must be able to compile it in less than 30 seconds and at most 512 MB of memory. No other extensions different than the ones listed above will be run.

FEEDBACK Interface

Contestants use the IOI Testing and Grading Web Interface to view the status of their submissions and, optionally, to view their results. When a task is submitted, it will appear immediately in the system, but there may be a few minutes delay before it is graded. Once it is graded, the contestant will be able to view summary results on a few simple "public" tests. If these results are satisfactory, the contestant may release test the submission. This shows the results of the submission on the official tests for all subtasks. To release test a submission, it is necessary to forfeit a "release token." Contestants will be given at least five release tokens per task, which may be used at any time.

Tasks are graded whether or not they are "release tested." If a contestant runs out of tokens, he or she will not be able to obtain feedback, but may continue to submit, and the submissions will count.

SCOREBOARD

Contestant scores will be available on a scoreboard that will be available to local and internet spectators. The scoreboard will not be available to contestants.

Assistance

Contestants may ask the support staff for assistance at any time. The staff members will not answer questions about the competition tasks, but will deliver Clarification Request Forms and printouts, help locate toilets and refreshments, and assist with computer and network problems. The only manner in which contestants are allowed to access the network is via web browser access to the designated Competition Server; even running a single 'ping' command is strictly prohibited and may lead to disqualification. Contestants should never attempt to "fix" or "debug" or even "check" computer or network problems themselves; instead, they should ask for assistance.

Ending the Competition

Three warnings will be given at 15 minutes, 5 minutes, and 1 minute before the end of the round. Each warning will be given by an audible signal. The end of the round will be announced both verbally and by an audible signal. At the announcement ending the round, contestants must immediately stop working and wait at their desks without operating the computers or touching anything on their desks. An additional announcement will be made instructing them to leave their tables and exit the competition room.

At the end of the first competition round, any previously submitted items a contestant would like to use during the second competition round should be left at the workstation. All other items should be taken out of the competition hall, including task statements. At the end of the second competition round, contestants should remove all items including their mascots and dictionaries or any other previously submitted item; nothing should be left behind.

Grading

For output-only tasks, output files will be submitted as described above and scored as described in the task description.

For tasks that require programs as solutions, the submitted source files will be compiled under Linux, enforcing the source file size, and compilation time and memory constraints. The RunC tool will will be used to compile solutions of tasks (say the task name is abc):

1. For C: runc abc.c
   

1. For C++: runc abc.cpp
   

2. For Pascal: runc abc.pas

Detailed feedback when submitting

For all tasks, detailed feedback is availble for a limited number of submissions. Detailed feedback will inform the competitor whether or not each subtask has been solved, and the score awarded.

If the subtask is not solved, summary information from RunC will name the test input file for which the submission failed, and the reason the task subtask was not solved:

1. Output is not equal to expected output
2. Run-time error
3. Output limit exceeded
4. CPU limit exceeded

No information on the actual data, the output produced by the contestant solution or any other execution details will be given to the contestant.

The contestant may deduce from the feedback his or her provisional score for the task. However, there is a small chance that the score may change due to appeals, or nondeteminacy of the contestant's submission. Nondeterminacy may arise intentionally due to the use of pseudo-random number generators, or unintentionally due to programming bugs or marginal running time. Submissions may be re-evaluated many times, and the final score will be that yielded by the final evaluation.

In the unlikely event that re-evaluation takes place during the competition, contestants will be notified by a pop-up message at their desktops.

Cheating

Submitted programs are not allowed to:

1. access the network,
2. fork,
3. open and create files,
4. attack the system security or the grader,
5. execute other programs,
6. change file system permissions,
7. read file system information,
8. make system calls.

Contestants must use only the workstation and account assigned to them on each competition day. In particular,

1. contestants must not attempt to gain access to root or any account other than the one assigned to them,
2. contestants must not attempt to store information in any part of the file system other than the home directory for their account or the /tmp directory,
3. contestants must not touch any workstation other than the one assigned to them,
4. contestants must not attempt to access any machine on the network, other than to submit tasks and to view submission results,
5. contestants must not attempt to reboot or alter the boot sequence of any workstation.

All of the above actions are considered cheating, and may result in disqualification.

Appeal Process

Submitted solutions are evaluated using data which conforms to the specification given in the problem statement, but which is hidden from competitors during the competition.

Provisional grades, based on these tests, are available immediately to competitors. A summary will be delivered to team leaders' mailboxes as soon as they are available, shortly after competition ends. The judge data will be made available electronically in the competition area after the competition as well. Contestants and team leaders may use the contestant's workstations to verify that the grades are assessed correctly.
A Team Leader may file an appeal by completing an Appeal Form, available near the mailboxes, and submitting the completed form to the mailbox labeled "Appeals" at least 30 minutes prior to the final GA meeting of that competition day. Every appeal will be reviewed by the Scientific Committee and the team leader will be notified the committee's decision. All appeals and their disposition will be summarized at the final GA meeting of that competition day.

In the event that a mistake is discovered in the grading of a task, every submission of that task will be re-graded and re-scored whether or not the scoring of that submission has been appealed. Note that re-scoring may result in a higher or lower score for any contestant. Should anyone's score change after grading results have been distributed, new results will be printed and distributed to them.

Medal Allocation

After the second Competition Day and before the IOI Awards Ceremony the medal distribution is determined by an automatic procedure, based on the number of points the contestants achieved. The medal awards are uniquely determined by the following rules:

1. The score necessary to achieve a gold medal is the largest score such that at least one twelfth of all contestants receive a gold medal.
2. The score necessary to achieve a silver medal is the largest score such that at least one fourth of all contestants receive a silver or a gold medal.
3. The score necessary to achieve a bronze medal is the smallest score such that at most one half of all contestants receive a medal.

Code of Conduct

Any contestant:

1. interfering with other contestants' activities,
2. damaging the competition or evaluation facilities,
3. accessing the network or any computer other than their own during a competition round by any means other than as directed to submit and view results,
4. subverting the security of any computers (e.g., making any material accessible to the network from their computer),
5. interfering with the execution of the competition in any way,
6. communicating in any way with anyone other than the competition staff during a competition round (whether providing or receiving information or materials),
7. or attempting to do any of the above

will be subject to disqualification. In addition, the submission or execution of programs as described on the the list above of prohibited program actions may result in disqualification.
Any attempt to gain a score other than by solving the tasks as specified, or to interfere with other contestants is a violation of the contest rules and may result in disqualification. The International Scientific Committee is the final arbiter of any violations of these rules.

It is of the utmost importance that team leaders communicate these expectations to contestants and to all members of the delegation.