How we process Researcher Project applications

Here you can learn about the administrative procedures for the Researcher Project calls. Go straight to:

Preliminary review and rejection of applications Assigning the application to a panel Recruitment of referees Impartiality  Mapping of expertise Information to referees

The panel’s tasks before the meeting The panel meeting Assessment of relevance Portfolio assessment Decision on allocation Feedback to the applicant

Preliminary review and rejection of applications

As soon as the application deadline has passed, the Research Council conducts a preliminary administrative review of the applications received. Applications that do not meet the formal requirements described in the call may be rejected.

Assigning the application to a panel

Applications are reviewed by a set of thematically organised panels of peer reviewers. Which panel an application is reviewed by is based on the topic the applicant has chosen in the call and the research content of the application. 

Each panel will cover a broad research area. The peer reviewers will have less specialised expertise for each individual application.

We use a machine learning algorithm to help us categorise the applications into different groups. The algorithm is trained with previous years' placement of applications in panels and is based on the text in the project description. The algorithm saves the case officers a lot of time by roughly sorting the applications we receive.

The case officers of each panel distribute each proposal to the panel that will give the best possible assessment. The project's title, objectives and summary are the main sources of information we use when assigning applications to panels. We strongly encourage applicants to write these texts clearly with case officers and referees with a general scientific understanding of the field in mind.

Many applications are considered for more than one panel. This includes interdisciplinary proposals and proposals where it is difficult to find similarities to other applications. The boundaries between each panel, can and will vary from year to year. It is not our preliminary classification before the application deadline that determine where the applications are placed, but how the applications best fit together, and how they are best covered by available referees.  

Recruitment of referees

Recruiting referees to assess the applications starts before the application submission deadline but is not completed before proposals have been assigned to a panel.

Our referees must meet the following requirements:

• They must work abroad.
• They must be active researchers who have produced a significant body of work in terms of both quantity and quality. 
• They should qualify for professorship. The minimum requirement is qualifications as associate professor.  

Expert Lookup is our most important tool to find referees. We perform automated searches based on project titles, objectives and summaries, as well as manual searches. The case officers then make a thorough assessment whether the referees are a good match for one of our panels, or perhaps to assess individual applications. We supplement the findings in Expert Lookup with searches on other websites such as Web of Science and Google Scholar, as well as well-known foreign universities in the various fields. The list of sources varies from field to field. We also consider the applicant’s own suggestions for suitable referees or description of suitable expertise.

The main rule is that the same referee is not to be used more than three times in a row.

When the entire application process is finalized, the panels with their referees are published.

Impartiality 

The researchers that have accepted to be a referee, receive a list of all project managers and partners of the applications assigned to their panel. The list is returned with statements regarding their impartiality towards everyone on the list. They are asked to pay special attention to the points in the impartiality provisions where collaboration, friendship and conflicts are discussed.

Referees do not get access to applications for which they are not impartial and do not participate in the panel meeting when these applications are discussed.

Impartiality in administrative proceedings.

Mapping of expertise

The panel must consist of minimum four referees. The combined expertise of the referees must cover the range of topics and diciplines covered by the applications, and along the axis from basic to applied research. At least one panel member must have experience from interdisciplinary research when such applications are handled by the panel.

Before the referees get access to any content of the proposals they are to assess, they must consent to our provisions on confidentiality. Based on the title, objectives, and summary of the applications, the referees declare their level of expertise. We encourage applicants to word these texts in a way that enables the panel members to state their level of expertise as accurately as possible. The texts must be informative to referees with general expertise in the field.

The panel members indicate one of the following levels of expertise on each application:

• Specialist (S): The proposal is within your primary area(s) of expertise or connected to your research interests. You are well qualified to evaluate the proposal.
• Generalist (G): You have a general knowledge of the main subject of the proposal (or at least one of the main subjects if there are several). You are qualified to evaluate the proposal
• Minor (M): You have only minor relevant expertise on the main subject(s) of the proposal.

According to our standards, for each application there must be least two panel members with generalist or specialist expertise. If the mapping of the panel’s expertise shows that some applications are not adequately covered, we will find additional external referees with specialist expertise. These external referees evaluate only individual applications and provide a similar assessment as the panel members.

Information to referees

The panel members receive information on how to process the applications. They receive informatin about the entire process and their role before, during and after the panel meeting.

The four assessment criteria (Potential for advancing the state-of-the-art, Quality of R&D activities, Impact, effects and Implementation) are also described in detail and the scale of marks is defined. We emphasise the importance of consistent marking and the quality of the feedback to the applicants.

The panel’s tasks before the meeting

All panel members read and assess all applications reviewed by the panel. The grant application form, project description and CVs form the basis of the assessment. Each application is given special attention by a principal and a second assessor.

• Prior to the panel meeting, the members submit their individual assessment of each application. This is made available to the other referees when everyone has submitted their assessments. On this basis they may prepare for the discussions in the panel meeting.
• The principal assessor submits a draft for a complete written assessment and a mark for each of the four assessment criteria.
• The second assessor submits a brief written assessment and a mark for each criterion.
• The other panel members submit a mark for each criterion.
• Any external referees who are not part of the panel will submit a complete written assessment and a mark for each criterion.

The panel meeting

The members attend the panel meeting with their different professional views, various emphasis on the elements of the application and their own interpretations of the scale of marks. All panel meetings start with a review of the general instructions, including how to use the scale of marks, the assessment criteria, and expectations related to the content of the written feedback to the applicant. The panel reaches a consensus-based assessment and a unified use of the scale of marks.

The Research Council's case officer will not participate in the scientific discussions. Their role is to provide guidance and ensure progress in the meeting, a joint interpretation of the assessment criteria and scale of marks, and that everyone contributes. They also handle impartiality according to the rules and stop discussions about matters that are outside the panel's tasks. The case officer reads the written assessments, making sure they are in accordance with the panel’s decision and meet the described quality requirements.

The panels have access to the priorities described within each topic in the call, which allows them to see the context in which the application is written. However, the panel will not assess its relevance. The application’s relevance to the topic, i.e. how well it addresses the priorities, will be assessed by the Research Council after the panel meetings.

Assessment of relevance

The relevance assessment is conducted by the RCN administration. As a general rule, only applications given the mark 5 or better on all criteria are eligible for funding and will be assessed for relevance to the topic selected by the applicant. For some topics, if the number of applications is very large compared to available funding, stricter mark requirements might be set for the applications that are to be assessed for relevance. 

The panel’s judgment of the application’s impact addresses the project’s general potential for scientific or societal impact. The Research Council assesses if the application addresses the priorities described under each topic in the call for proposals. These priorities are based on the ministries’ Letters of Allocation to the Research Council.

Primarily, the attachment Relevance to the topic is the basis for our assessment, but the application, project description and the referee panel's assessment may also be used. For Researcher projects we have internal guidelines for assessing relevance to the topic (only in Norwegian).

Portfolio assessment

When the portfolio assessment begins, the applications have been awarded five separate marks and written assessments: Potential for advancing the state-of-the-art, Quality of R&D activities, Impact and Implementation and Relevance.

When we prioritise proposals and prepare ranked lists for the portfolio boards, we carry out a portfolio assessment that includes:

• The assigned marks based on the assessments
• A good distribution of projects in relation to priorities set out for the specific topic
• Any changes in the financial or scientific framework set by the ministries
• Priority will be given to projects with female project managers when the applications otherwise are considered to be on a par
• All these factors are included in the portfolio assessment, which forms the basis for the ranked lists we present to the portfolio boards.

Decision on allocation

The portfolio boards decide which proposals that are granted funding. There are 11 portfolio boards with significant variation in available budget, number of announced application types, number of Researcher Project applications for processing, thematic span and expected number of granted applications.

The various boards may emphasise the factors mentioned above (see the section ‘Portfolio assessment’) differently. However, they are not allowed to deviate from the priorities described under the topics of the call. Commonly, the administration presents alternative ranked lists to the portfolio board where the factors are differently emphasized. The lists form the basis for the portfolio board’s discussion of the considerations they want to prioritise, within the available budget and the priorities described in the call. Their final decisions may differ from the lists presented by the Research Council.

Feedback to the applicant

For applications that are not grated funding, the applicant will receive a letter of rejection in MyRCN as soon as possible. All applicants will receive feedback from the panel in terms of marks with written assessments on the four criteria. For applications assessed for relevance the written relevance assessment will be included in the feedback.

The letter of decision will also provide information on

• Which referees participated in the panel
• Which portfolio board made the funding decision
• The principles applied in the portfolio assessments
• Statistics about the applications and marks
  
  

We will publish on our website and in our newsletter which applications are granted funding.