Establishing and Supporting CERTs for Internet Security

4 September 2014 - A Best Practice Forum on Other in Istanbul, Turkey

Also available in:
Full Session Transcript

***
This is the output of the real-time captioning taken during the IGF 2014 Istanbul, Turkey, meetings.  Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the session, but should not be treated as an authoritative record.  
***
 
   >> MAARTEN VAN HORENBEECK:  Good morning, everyone.  And thank you very much for attending the Best Practices on Establishing and Supporting CERTs for Internet Security.  My name is Maarten Van Horenbeeck and I am the President of an organisation called FIRST, Forum of Incident Response and Security Teams.  And I am together with two other lead experts that were invited to run this session, run the preparatory process, finally get to deliverables.  And those people are Adli Wahid also of FIRST.  He is our outreach liaison and Christine Hoepers from CERT.BR.  In addition we have Wout de Natris who will be our note‑taker today and who is actually the consultant who has been compiling all our findings in our document which is now open for comment on the IGF website.  
To really give a quick introduction in the process that we will follow because we have a short amount of time to cover quite a bit of work that happened in the last few weeks.  We will start off with a quick introduction to set the stage for individuals attending that are not familiar with the CSIRT community.  And Christine will give an introduction at a high level.  So we will delve in deep after that.  And we will also look at the stakeholder groups and finally have a look at the document today.  
We have a set of slides that summarizes the document.  Important to note that the wording is not 100% identical to the document.  That's not possible within the space that we have in the slide decks.  So what we will do we will walk through those slides, provide some comment and then open it up for comment from all the people that are attending here today.  All those comments will be taken in to the final document.  
One thing that I would like to note is that we will be very careful on how we spend our time.  So that means we will go over certain topics fairly quickly.  It is important for you to realise that we want to be heard.  If you don't have the opportunity to comment in this session, please go online.  The document ‑‑ the period for comments is now, open until end of day tomorrow.  There is some discussion about extending it, but that hasn't been confirmed.  Please now put in any comments on the document.  
    Now to get started very quickly could I have a show of hands here from who is actually from the Government stakeholder community?  Excellent.  So we have a few people from the Government community.  Who here represents Civil Society?  Excellent.  Who represents the technical community?  Excellent.  And finally private sector, do we have representatives from the private sector?  Excellent.  So we seem to have a couple of good groups here for the discussion.  To move in to the discussion, one thing I'd like to understand from each of the communities what are really the outcomes that you would like to see from this process, and are there any comments or concerns around the current process that is being followed.  It would be great to hear from one individual from each of the communities.  Maybe from the Government sector, from the Government community.  Anyone that would like to comment on the outcomes that you would like to see from this process?  There are no opening comments from the Government stakeholder community.      
   >> PATRICK CURRAN:  Yes.  Maarten, can I ask, are you regarding the EU as a Government or what?  
   >> MAARTEN VAN HORENBEECK:  I think that would be work for the Government community.  If you would be willing to contribute something I would appreciate it.
   >> PATRICK CURRAN:  My name is Patrick Curry and we have spoken on the phone.  I chair two subgroups.  Instant notification, and information sharing and on risk management and that involves organisations like CERT EU and Governments and industry.  And I think at the very high level what's really coming through is the requirement for collaboration in a major way and for the mechanics to be in place for risk assessment, intraoperability and for trust, the trust mechanisms that we would understand as AAA.  And so there is a lot of work in that space.  So it is building, if you like, the foundations on which CERTs can work together.  There is a very strong demand for that.  
   >> MAARTEN VAN HORENBEECK:  Thank you very much for that contribution.  Yes.  
   >> JORDANA SIEGEL:  I thought I would just add something from a Government perspective as well.  I am Jordana Siegel.  I am from the Department of Homeland Security in the United States which includes the computer emergency readiness team in the U.S. and as well as our ICS CERT control systems.  And in terms of outcomes from the session I would just say that I think that one thing that would be very valuable from our perspective is thinking about how the multi‑stakeholder model can enhance the work that CSIRTs do around the world.  Certainly CSIRTs are not Governmental.  They are industry.  They are academic.  They are otherwise, and I think that just thinking about how the stakeholders that come to IGF can work together to better enhance the type of work that CSIRTs do, would be really beneficial.  
   >> MAARTEN VAN HORENBEECK:  Thank you very much.  Are there any contributions from Civil Society?  What would Civil Society like to see come out of this best practices effort?  Yes.  
   >> AUDIENCE:  Sorry.  I am Dr. Moral.  I am from the Francophonie group here.  I am at the same time the President of the Lebanese Technology Association and a member of the cybersecurity panel at the World Federation of Scientists and at the same time I am a member, founder for cybersecurity.  
    Well, I would like to know or to see at this session how we can as Civil Society really instigate the awareness, the common work, especially in the Arab world for a better management of cybersecurity risks and threats.  And actually I would like also to know if there is any chance for CSIRT, privacy CERT.  Because you know in ‑‑ we don't have a CSIRT in Lebanon.  They are working with the ITU to create one.  But we do ‑‑ we do think that maybe the private sector won't be so trustful to work with Governmental maybe.  Maybe.  I'm not sure.  I'm saying maybe.  So it could be a good idea to have a CSIRT, private, yes or no.  That's it.  And because we work on capacity building actually for cybersecurity, for CSIRT as an Association we have cooperated with the Francophonie on a seminar for the Ministry of Defense in Lebanon on creating a CSIRT.  So we are active and we want to know what we can do to enhance cybersecurity.  
   >> MAARTEN VAN HORENBEECK:  Thank you for that contribution.  I hope the document will answer some of those questions.  We look very much forward to your participation and the discussion of it.  Moving on to the technical community, is there any input from the technical community on what they would like to see from this effort?  Yes.  
   >> AUDIENCE:  I am Paul Vixi from Far Side Security.  The CERT community, the national community of CERTs is a big part of the market for a lot of new startups in the cybersecurity industry.  A lot of what's being told is joint.  And I have found in my dealings with the CERTs across the world that the older ones ‑‑ the newer ones are often wondering about the difference.  I am hoping from the outcome from these documents that you are preparing that we are going to level set across the industry and keep your average PC from the security company from being able to succeed on training of the ignorance of new CERTs.  
   >> MAARTEN VAN HORENBEECK:  Thank you for that input.  Moving on to private sector.  Is there anyone from private sector who can share some thoughts on preferred outcomes?  
   >> AUDIENCE:  Yes.  I guess we are speaking to the community.  I am sorry, I run two collaborative organisations which are international and they have industry and Government involvement.  So I will offer two points.  The first is on CERTs.  I completely agree, the problem of expanding CERTs in to lesser development ‑‑ developed communities and the requirements for the coordination and help.  And for that I give an example of the ANISA review that went from 42 CERTs to 220 CERTs in the space of three years.  Now this begs a question which is what is a CERT, because CERTs are very different in their character and nature depending where you look.  And we don't have enough terminology to describe this.  
The second point that goes with this is that we are already finding a very strong linkage between CERTs in their operation at different levels and counter fraud in particular and cybercrime.  So this is bringing in a far greater degree of law enforcement and human actor behavior and also a link out to crisis management, and we've had this particularly in their traffic management, for example.  
   >> MAARTEN VAN HORENBEECK:  Thank you very much.  I would encourage anyone as the discussion continues to take in to account these needs from the different stakeholder communities recognizing that they may not be comprehensively covering each community but take them in to account as you provide input or questions.  I want to give Christine a quick opportunity to sort of give a level set of what we are covering and what the CSIRT is like.  
   >> CHRISTINE HOEPERS:  Good morning, everyone.  I am Christine.  Before we work in to the document one of the things that we thought was beneficial is to try to define what is a CERT, what is a CSIRT.  So this is a classic definition that it is like one of the oldest ones.  This is why we kind of stick to it.  But really a CSIRT, it is not necessarily a company or department.  It can be both.  It can be for not‑for‑profit, internal or external, serving a region or country, a small organisation, a university.  But the basic point is that they would receive notices about incidents that could be coming from appliances or coming from mail.  Could be coming from a call.  You are noticing that you have incidents.  You analyze and help the organisation to respond in the best manner possible.  So this is I think the best definition of a CSIRT.  This is why people want to have this narrow definition and it usually gets a little bit broader because CSIRTs serve diverse communities and these communities is defined as a constituency.  This is a word that we will be saying here today a lot.  So this is why we thought it would be nice to define.
So a constituency is basically the public that we serve.  So that could be the customers from your CSIRTs, that could be citizens in a country, users of a specific service, employees of a company that basically is the constituency.  And when we are talking about an incident it is very technical in nature, and usually it is really any suspect behavior or any event that happens in the network that could affect its security.  Not necessarily we are only dealing with the text that actually were successful in some way.  Sometimes all our work is to see the hints, to see what's going on and try to prevent incidents from happening.  So there is a lot of proactive and reactive things going on.  
    When I am discussing later on the definition of the issue we are going to go a little bit further in to services.  But I think it is important to know that we are dealing with incidents, with the facts, usually technical in nature or that could be from behavioral people, from doing something wrong or configuring something wrong in a network, but we need to help these people to understand this incident and to respond in the best way possible.  
And just to set a little bit on history, it is always good to remember that a first team that was created was the CERT coordination centre and it was created in reaction to the first big incident we had on the Internet and that happened in 1988.  And that was when we had the Morse war that stopped actually 60% of the Internet at the time.  That was the point that people come together and said this is probably going to happen more frequently in the future.  So how can we address.  So this is how it started, purely reactive, but it evolved a long time to having CERTs ‑‑ CSIRTs in different organisations, to having computer security instant response teams serving distributed communities because the Internet is distributed.  
So you need to have those team as close as possible to where you are having incidents and where you have people and metrics needing help and all these teams they work in cooperation with the network.  There is a hierarchy.  It is based on trust and teams knowing each other's missions, knowing the technical competency and knowing that they can trust confidence about attack, techniques, about threats, about what is going on.  And this helps each individual organisation to improve its own security and as a larger goal it helps the whole Internet as a system to be secure.  And we will be talking about cooperation and team.  And we thought it would be beneficial to have this first terminology and organisation here.  Before we move to the document, we thought we would give time, there is a lot of Forums where CSIRTs meet and these Forums have different reasons why they were created.  And we would like people here who are present in these Forums to speak a little bit about them.  
   >> MAARTEN VAN HORENBEECK:  Thank you very much, Christine.  So as Christine mentioned we are lucky to have a lot of people here with a wealth of experience in that wider community.  And I'd first like to call Yurie Ito who is the Chair of the CERTs to briefly introduce the organisation.  
   >> YURIE ITO:  Yes.  Good morning.  My name is Yurie Ito.  We have 22 teams from 19 economies get together and work together to ‑‑ for the regional level of cybersecurity risk reduction.  We are focusing on and, of course, we to instant response to systems, each other.  Help each other, share information to remediate the threat, but at the same times we focus, clean up Botnet, clean up machines infected, removing Botnet and try to raise cyber hygiene in each region.  And then we share the goals and we defined the shared goals that we are trying to make the regional and cyberspace cleaner and safer and more resilient.  
    So we start ‑‑ we have an annual general meeting.  Once, yeah, a year we have a face‑to‑face meeting, and then we have a safe secure mailing list providing the infrastructure for the members to communicate and do instant response.  
    Also we have a point of contact arrangement.  They are economies which has more than one member from one economies.  We asked each economies to assign one point of contact.  So when we have a severe incident we know where it is and in these teams or arrange to be providing, be able to provide the fastest response.  So that's why we call it a point of contact arrangement.  We conduct a cybersecurity drill once a year.  Also we share the network monitoring sensors and that's also realtime data sharing with the member teams.  
    We do have the multi‑stakeholder approach working with the Intergovernmentals.  This AP CERT itself is the CERTs community that we work with, the APNIC Tel and other Intergovernmental organisations, providing then the technical insight, what's the problem, what's the threat and that type of information.  We work with the Asia‑Pacific Internet community, Apricot and the APSTAR community.  
    So ‑‑
   >> MAARTEN VAN HORENBEECK:  Next is Africa CERT and John Bera of African CERT had initially planned to be here but he couldn't make it.  On the conference on Internet Governance under the team administration of Internet protocol addresses and the domain names and strengthening African emerging institutions in Kotlu December 15 to 17, 1998 African pioneers called for uniting the African Internet community to the establishment of several organisations to tackle specific issues.  The organisations are known as African organisation of Internet Governance and among them Africa CERT facilitated by the African network operator group.  The regional Internet registry, the Association of African Universities the African research and education network with the support of JPCERT and APCERT.  
The objectives of Africa CERT include but not restricted to coordinating cooperation among CSIRTs and assisting African countries in establishing CSIRT and fostering and supporting education and outreach programmes in ICT security, strengthening the relationship among CSIRTs in Africa and with stakeholders across the world, encouraging information sharing, promoting good practices and experience, sharing among members to develop a comprehensive framework for cybersecurity and assisting African CERTs in improving cybersecurity and ICT infrastructure, and finally promoting collaborative technology, research, development and innovation in the ICT security field.  And then there is the organisation of the Islamic cooperation CERT, and Adli can tell us a little bit more about what they do.   
   >> ADLI WAHID:  The objective is to provide a platform for member countries to explore collaborative initiative and possible partnership in matters pertaining to cybersecurity.  It is more like an Association CERT.  However the OIC‑CERT is a legal entity which is attached to the organisation of Islamic cooperation.  And they were established from a meeting in Uganda in 2008.  The OIC‑CERT currently has 22 members ‑‑ 21 members I'm sorry.  And their activities are almost the same like APCERT which they have an annual meeting, cybersecurity, exercises, training and other than that as and when there is an incident in each member country they communicate and share information with one another.  Thank you very much.  
   >> MAARTEN VAN HORENBEECK:  Thank you, Adli.  Christine, can you share a little about what LAC CSIRTs does?  
   >> CHRISTINE HOEPERS:  In Latin America we have an initiative that is bottom‑up.  It is an initiative that is being sponsored by LACNIC and LACNOG.  And LACNOG is an African operators group.  This initiative, it is actually instead of creating a formal organisation to decide topdown we are going where the people are doing the map, where they are meeting a LACNOG and LACNIC meeting to bring the message.  They asked for a better Forum to meet.  LACNIC is helping us every meeting twice a year to meet all the teams from the region and this is not only for formed teams.  We have this meeting where anyone that is interested in learning what is a CSIRT and interested in forming their own CSIRT they can meet with the CSIRTs from the region and learn from experience and try to identify what are the common goals and what models that can work and the differences between services.  
We are going to our seventh meeting in October together with the LACNIC meeting.  In the last meeting we had people from 16 different countries and we have teams from academia, from the financial sector, from ISPs, Government and military attending those meetings.  We maintain a mailing list that is not only open to people who go to meetings but anyone working with incident response.  We are hosting a monthly web meeting that is getting people together to start talking about sharing information, sharing best practices that apply to some regional context.  And the idea to collocate these meetings with LACNIC and LACNOG is because all these people are already going there.  We have like costs problems because we actually just can't leverage from people already traveling in the meeting, and this is a way to reduce the cost of getting CSIRTs together.  And also get the whole community and other stakeholders to understand what we are doing in the region.  So this is basically how CSIRTs is organised.  We go from very informal to a point we will get formal in the future.  It is a complete bottom‑up and stakeholder group that is meeting in the region.  
   >> MAARTEN VAN HORENBEECK:  Thank you very much.  Finally on this deck you can see these other organisations.  And I do not believe we have any representatives from TFC CERT or EGC CERT.  TFC CERT is an organisation of CERT teams mainly in Europe and they operate a programme that is called Trusted Introducer which is a way for CERTs to be accredited or listed based on the existing relationships that they have within the community.  EGC CERT is a fairly unique organisation because they are a closer knit organisation of CERT teams across Europe.  And then finally FIRST, the Forum of Incident Response and Security Teams is a global organisation that aims to bring together security teams based on a vetting process in which teams are essentially reviewed by other members to see if they meet the minimum requirements to join that community.  And once they join they benefit from the ability to share best practices, share experiences, work on standards together.  And the organisation also organizes conferences and symposia across the world to bring the CSIRT community closer together.  
Moving in to the bulk of today's session what we will do next is as you are aware over the last six to eight weeks there has been a discussion taking place on a mailing list which was organised by the IGF and on that mailing list several people from various stakeholders, communities have been able to share ideas and thoughts and concerns that they had with the way the CSIRT community works today and ideas on how CSIRTs can better work with other parties in the stakeholder communities.  
    Now all of that work has been assembled in to a document which is today published on the IGF website.  It has been open for review I believe for the last two weeks.  What we will do today we will walk through some of the entries in that document based on the outline as defined by the Multi‑stakeholder Advisory Group.  Open for some very brief comments from the participants on each of those sections, and we will note the comments and there won't be time to open it up for wide participation simply due to the time limit we are dealing with today.  If you are unable to get your input in the session today, please go to the website and provide your input and they will be taken in to account as the document is being assembled.  We will start off with the first major section of the document which is definition of the issue, and Christine will be guiding us through that section.  
   >> CHRISTINE HOEPERS:  Thank you, Maarten.  I am going to try to highlight what's in the document, especially for people that didn't have time to read the document or that don't remember everything that's there.  And then we are going to open for comments for any issues that people have.  So our document starts defining what is a CSIRT.  And as Maarten said this document, it actually gets all the input that was discussed in the mailing list and submissions that were sent.  In our discussion it was very clear that all people contributed that there are very different types of CSIRTs in many different organisations with different objectives and with different naming.  So naming is not a norm.  It is not ‑‑ you don't need to be called a CERT, a IRT and a CSIRT.  What is really important is the community you serve and the services you provide.  
    This team as I said before they are within Government organisations, academia, private organisations, not‑for‑profit organisations.  Like JPCERT is a good example.  It is a not‑for‑profit that is also a national team.  And they come with all varieties.  These organisations work for their own constituencies.  There are some companies that provide CSIRT services but there is ‑‑ the main Consensus is when you have a response team it is a team of experts that responds to incidents that coordinates the Resolution.  What that means is that you get everyone that needs to be involved in to solution, to understand the problem, to do their own partner to it and try to coordinate efforts to mitigate a threat, to respond to an incident and to try not to repeat the incident in the future.  And it was also a Consensus that CSIRTs are involved a lot in the technical communication and in trying to improve risk reduction and Internet health.  So this is where CSIRTs have been moving more in to the proactive part of their work.  
    So another ‑‑ a lot of discussion was should we call a CERT a CSIRT and what is a difference between a computer response team and a computer security response team.  And a more generic neutral term that represents better what teams do is CSIRT.  That actually deal with incident response because depending on the context that you are, the word "emergency" is sometimes too wrong because we are not necessarily dealing with things at times that could be perceived as an emergency in some communities.  And this sometimes leads to a misunderstanding.  It was after this discussion that the whole group decided that we should keep CSIRT as the term, Computer Security Incident Response Team because that better reflects the nature of the work dealing with incidents of computer systems on the Internet.  The other part, I need to ‑‑
   >> MAARTEN VAN HORENBEECK:  Christine, can I ask, do you want to have any input or do you want to have questions at the end?  
   >> CHRISTINE HOEPERS:  For every session we are going to go to the end and then we are going to open because otherwise we can run out of time and not have time to cover everything.  As Maarten said we could have the afternoon, we could have the whole week to discuss this.  And another discussion was a lot of questions about whether national CSIRTs, whether national teams and what are teams of last resort.  We tried to capture in the document as best we could that discussion would happen with the mailing list.  One Consensus a CSIRT of last resort is a CSIRT that someone can turn to when you find no one else.  So that happens a lot in our day‑to‑day lives.  Like I have a niece in Japan.  If I know a CSIRT in Japan that I can contact directly that would speed up mitigation.  Sometimes it is a metric that I don't know.  So I can go to JPCERT that also acts as a CERT of last resort and ask them to help us find the right person.  And also most of the times national teams are also teams of last resort, but it was brought up in the discussions that more and more we are having more teams for metrics of national importance for critical infrastructures, for sectors and there is no right or wrong.  Not all teams are maintained by the Governments.  Not all teams have the same mission or not all of them provide services to infrastructure.  
This is a mix and from what people contributed it appears that all of the models have benefit and disadvantages.  But usually the decision on how to set up a national team it is based on local culture, based on who has expertise and who has funding to create a national team.  And there are several factors that impact on that decision.  So the experience of the teams that we're discussing there, national teams and other organisations that work with national teams that experience was that each country needs to identify what works best and this is what was the discussion about national teams.  
    And last two slides, like we discussed about services.  We didn't put too much detail in to services because you see the document that in these documents describing a nation centre and there was a document that was written by TCF CERT from Europe and everyone that describes at length.  So we pointed from evolution, we have reactive services.  That's how it all started in '88 that we will have an event, you have an incident and we have an attack and you have to react quickly, because as quick as you react you have less damage.  You have less impact.  And you can't really make the problem not be so bad.  But more and more we have teams working proactively because CSIRTs identify the threats and they see what's more common.  And usually when you are handling an incident and responding to an incident you can identify the root cause.  And then you can try to work with all the stakeholders to try to address that for that incident not be repeating itself a long time.  So the proactive service they involve interest awareness to helping organisations to better set up systems.  And there are some teams that have what are being described as security quality services that are services to augment other services that could be risk analysis, could be security of systems.  So it is a mix of services.  
And there is also ‑‑ the point that as we have diverse teams, some teams provide services to their own community but most of the teams also have a lot of what could be perceived as services when we cooperate with other teams.  Usually you share data about new threats and you have like data feeds from organisations.  So if there is some information that goes on that could be also perceived as a service.  There was a discussion and a lot of questions about tools.  What are the tools that CSIRTs need, and I think it was a Consensus that the tools are related to the services you provide.  At least you need some tool to kind of track incidents, to have some statistics to know what you are dealing with, but the tools will be linked to services that you provide and some discussion was brought that maybe dealing with data feeds would be seen as a tool, and for people that do not use data feeds we have a lot of people on the Internet.  And you can share that information with a metric owner usually to help them improve their security and notice that they are generating incidents to the Internet.  Share data feeds for cleaning computers or working with other organisations.  And last point it was discussed before we go for discussion for defining the issue is industry cooperation.  There is a lot of cooperation going on.  
People thought that in some specific industry sectors ‑‑ we have seen some countries that don't have any resources and you have ‑‑ even though you have several organisations in one sector that could be competing against each other, financial, telecommunication they decided to join forces and create one team for the whole sector.  So we shared some ‑‑ there was some examples shared.  In other countries where economies we have the model of information and analysis centre.  There is the ICET model.  Where usually the organisations themselves have their own teams but they get together in a Forum to exchange information pertaining to that specific sector.  You can have a financial sector, critical infrastructure, telecommunication.  It all depends on the country you want.  And most of the privatization they can operate their own team and cooperate with other organisations.  There is a lot of discussion with questions about management strategies, but of the Consensus was that there is a lot of privatization.  In the document we have links to APCERT, to JPCERT, to all best practice about how to set up and manage your team.  At the end of the day we put this load of information in the document which is pointers to other documents.  So this is an overview of the defining issue and I think we would like to open the floor for comments or issues.  
   >> MAARTEN VAN HORENBEECK:  When you provide a comment, any type of comment is very welcome.  One thing that I'd specifically like to ask you to think about is what other opportunities do you see here for where other stakeholder communities would value becoming involved in this type of effort.  Because a lot of this work actually does come from the technical community because that's where most of the CSIRTs operate.  At the same time there is a lot of opportunity for other communities to provide feedback on these issues.  So if you see opportunities for where the discussion needs to deepen definitely share those.  Outside of that we are open for comments on this first section out of a total of six that will come ahead.  Sorry.  Did you have a comment?  
   >> AUDIENCE:  I have a question.  
   >> MAARTEN VAN HORENBEECK:  Oh, please go ahead.  
   >> AUDIENCE:  Because following the ‑‑ what Christine was saying we have this feeling that we can create a response team and we can call it whatever we want.  Is that true?  
   >> CHRISTINE HOEPERS:  Yeah.  Basically what happens is that the name is not really the most important thing.  We have seen a lot of confusion with people, call it an incident response team, a CERT, a CSIRT.  Basically stress that CERT is a trademark in the U.S. and Europe.  The reasons for creating the trademark were because some people are abusing it and trying to sell services that were not good.  But really you are doing incident response.  So how you call it is not really important.  At least it was never important for the technical community  and has never impeded anyone from creating a team.  What we see happening is that sometimes as the community at large in a specific country, for example, recognizes the term CERT, then it is easier for the people to understand what you do because you call it CERT.  Like in Latin America most of the teams are just calling themselves CSIRT but because the term is very recognized in the region.  I think it all depends on what you are doing, but these terms are interchangeable, most of that.  I think that Jordana has a comment.
   >> JORDANA SIEGEL:  I just wanted to say from our experience in the United States even in my organisation we have two CERTs as I mentioned before and one is the United States Computer Emergency Readiness Team, because a decision was taken when the CERT was formed that it wasn't just about response but also prevention.  And so it wanted to reflect both of those concepts in the name, but that was a decision of the director at the time.  And then the ICS‑CERT, which is the Computer Emergency Response Team.  That's a different name although part of the same organisation and part of the community of international CSIRTs with the same functions and services that Christine described.  And so I think it is important to have that identifier so you know that you are part of that community, but I think the specific terminology is something not worth getting focused.  Sometimes it is important but sometimes it can be a rat hole for discussion.  It is sort of the commonality of terms and then the specific function.  
   >> CHRISTINE HOEPERS:  You can have a concept to have one team or a hierarchy.  In Brazil we have more than 40 CSIRTs establishing and they are from all sectors and some of them call themselves CERTs, CSIRTs.  Some of them call them security and response teams.  There is a wealth of acronyms and names that you can use because it all depends on your culture.  
   >> AUDIENCE:  So I echo that one, if I may, because we have seen particularly out of the military community collaboratively.  We have seen the requirements to get some kind of command and control structure to enable coordination across organisations for response purposes.  And this goes from the complete tactical level right to the bottom where we used expressions like warning points and exactly as you said CERTs, CSIRTs and so on.  What's happening inside organisations now this delayering is getting more complicated because we are seeing breach response teams from response companies which are going in to help as a first aid squad when something goes wrong.  And because they are providing cyber insurance they are doing a complete risk assessment of an organisation both to cover the insurance but they also then provide the service once an incident happens.  Neiman Marcus is a very good case in point.  It had a breach response team that managed all aspects including the public engagement of the incident as a whole.  The character has become more complicated because it has to deal with business risk at the same time as it is trying to deal with an attack.  And you need to sort of call Ghostbusters at this point to come in.  And the situation is getting more complicated and it is a rainbow of names out there.  And it is trying to understand where this sits.  
   >> MAARTEN VAN HORENBEECK:  We have time for one more question on this.  And I see that you had something else.  
   >> AUDIENCE:  The question ‑‑ I ask this question because I want to know if there is any application or for the cooperation that the ‑‑ whatever you would call it, will have.  Because we know that CERTs or CSIRT or whatever maybe has to be accredited.  It has to be ‑‑ to cooperate maybe with law enforcement across borders.  So what I understood, what I hear and, I may be wrong, but it seems like we decide what we want to do according to the contract we create.  An according with the bylaw we have of the CERT or the institution and while I don't know, I had this idea that such organisation has to be trustful, has to ‑‑ has to know the standards, whatever because I am thinking of the legal aspects.  
   >> MAARTEN VAN HORENBEECK:  Maybe one thing that is interesting in this respect is not necessarily think of a CSIRT as a name but a set of services that it provides.  And one thing that is fairly universal in the community is that CSIRTs tend to publish a document which is called an RfC 2350 document and that document describes all the basic functionality of the CSIRT, the services that it offers.  If you are going out to work with a particular CSIRT, you can usually find that document on the website or published elsewhere.  It can describe what the CSIRT really does.  And many of them, actually they require the CERT to list out the type of services they offer.  So I think it might make more sense to look at the functionality rather than the name.  Don't know, Christine, if that covers ‑‑
   >> CHRISTINE HOEPERS:  Yeah.  
   >> AUDIENCE:  When you say that there is a requirement, there is some conditions, so the nomination or the name counts because I cannot say I am a CSIRT when I'm not answering what is required for me to be a CSIRT?  
   >> CHRISTINE HOEPERS:  In that level, yes.  Just I think to sum up when we were talking about the session, I think one of the discussions to me really makes it clear that really is a Consensus that if you put the word "emergency" it mixes things.  There was a lot of discussion that we didn't hear but were in the document that it is really important for confusing national strategies with CERTs.  Not necessarily you need a CERT like the one doing Army.  We have people using the term and this is what in the document take incident response team and not emergency response.  Because this could be conceived completely different.  So I think this enforces for me what we were trying to get the scope in this best practice.  We are talking about best practices for incident response and not best practices for national cybersecurity.  Just try to move for this mix of things here.
   >> MAARTEN VAN HORENBEECK:  Thank you, Christine, and thanks to both commenters.  And that's good feedback.  And please continue that conversation both on the mailing list and in the review website that has been published by the IGF.  Then we will move in to Section 2.  And Section 2 is on regional specificities and it will be Adli who will be driving this section.  
   >> ADLI WAHID:  Yes.  This is very technical.  One of the sections that we have in the document, and I hope you will have a look at the document, is regional specificities.  This is basically a section where we tried to collect information from people who participated in the process earlier on if there are certain trends in each region with regard to CERT in terms of capabilities, in terms of cost, in terms of legalities and things like that.  Any information related to that and the whole idea is to be able to see and get some inputs on whether or not there are certain things that are different from each region or if there are opportunities for further improvement and things like that.  
So one of the key points that was discussed within that group was that when we talk about cybersecurity incidents it is accepted by a lot of people, everyone in the CSIRT community that when you have an incident you need collaboration.  You need to work with more than one entity.  So it is sometimes not just point to point with another organisation.  Sometimes it is need to work with a lot of people in different places and so on and so forth.  And this somehow leads to some expectation that we are a other party, other team, other entity have the common, common capabilities, common skills to deal with that particular incident or to have a process in place of tools or relevant tools to process the incident or to handle privacy of the incidents and so on and so forth.  There are some expectations of the common understanding.  If you declare that you are doing CSIRT services these have a set of things that it is sort of expected that you have in place.  And when we have cybersecurity incidents it is also ‑‑ it was also discussed that sometimes not everyone gets the same incident.  And therefore information sharing is very critical.  
And although there are a platform for doing so in terms of sharing information, are still people in the group that believe that one of the biggest challenges is making this knowledge global, mean that making it scale.  And also when we have to deal with many entities in different locations and different regions then trust becomes an issue.  It is not stuff to just exchange information via e‑mail.  Sometimes there is a need to meet face to face first time and have some sort of discussion and do some other thing that CERTs exchange.  Anyhow we will get more in to that later on, but for this particular section we also have some representative from some region.  And we would like to invite them maybe to share some of their observations in terms of the trends or the features of CSIRTs in their region.  So the first one that I have on my list is again Ms. Yurie Ito from APCERT.  Maybe you can share some observations of CSIRT in that region.  
   >> YURIE ITO:  So CERT teams in Asia‑Pacific very, very different.  That's probably the characteristics of this region.  And then probably same in the other region as well.  But the teams are coming from the different culture.  Of course, very, very different culture, political system different.  You know, social, different social systems, technical level is different.  Team's authority is different.  Some of the team has authority to ISPs to do something.  Some of the teams do not have, for example, JPCERT, it is just a coordination centre.  We are requesting, we are asking for the cooperation but not having the enforcing authority.  So authority is different.  Different model of operation and funding model is different.  Size of the CERT is very different from 200 to, you know, two, three people CERTs.  So it is very different.  
I think one of the important things is knowing the others know well.  That's why we have a face‑to‑face meeting, what type of difference knowing the other team's authority and capability rules but most importantly I think for gaining the trust is based on operation and then know‑how safely to handle the sensitive information or incident threat, that type of information.  
So knowing the rules of the CERT community and those things are all in the CERT training material.  And this region we are actively doing a CERT capacity building training and also once it is established we will keep having a CERT training workshop.  So through those type of things we will make sure we understand each other, we understand the rules and so on.  So that's the trust building and there's accreditation and vetting system in the CERT community.  And you know the existing CERT, a couple, three CERTs need to nominate you to be a part of it.  So you have to start operating and gain trust.  So that's ‑‑
   >> MAARTEN VAN HORENBEECK:  Thank you very much.  Next I would like to call on Christine maybe to share a bit on the South America perspective.  
   >> CHRISTINE HOEPERS:  I am going to jump to challenges.  I think one of the challenges that we are seeing most of the Latin American and Caribbean regions is issues with funding, with prioritizing creation and formalization of CSIRTs.  Most of the organisations already do incident handling in an ad hoc manner.  And a challenge is that usually people think that they need to create this really big organisation that's very complex, but what they really need to allocate one or two people to do this full time.  And once they understand that it gets to a more mature level and the next level is how you incorporate.  And the only way to have meaningful cooperation is knowing each other and going to places, meeting each other and gaining trust.  There is no trust through a contract or ‑‑ it is really trust that you are doing the work well.  That if I share information with you you are not going to share if it is not appropriate for anyone else and that you have the capacity and technical expertise to understand and to help the community a lot.  
So I think this is one of the ways why we are actually just fostering having meetings and having people in the room twice a year at least meet each other to know each other's challenges.  One thing I was going to say we are going to our seventh meeting.  The first three meetings we were championing for people to say something.  Last few meetings we were extending meeting time and we were having lively discussions and that took time for people to know each other.  And now getting ready to do something else.  It is really a process.  So I think this is what we are seeing in the region and it is good that the teams are very diverse and they are very small.  You see small teams trying to leverage each other and to build trust in the region.  
   >> ADLI WAHID:  Do we have anybody from Europe?  Okay.  I will skip to the African perspective and Maarten will share something.  
   >> MAARTEN VAN HORENBEECK:  John Bera of Africa shared a view.  He noted that currently existing CSIRT teams cover less than two thirds of the African Internet ecosystem and also that they face the following challenges, a lack of tools to facilitate information sharing, a lack of standards, lack of training, some trust issues, competition within and between countries, resource battles in terms of funding, language barriers across the continent, cultural barriers and legal issues and also some of the issues that they are trying to tackle in their region.  
   >> ADLI WAHID:  Thank you very much.  And then we got to move on to the other section.  It is still the same section but on some of the topics.  One is whether or not there are entities out there who encourage their membership to establish cybersecurity strategies or incident response capabilities.  So it turns out there are.  So there are some from ITU, EU, African Union and also at the country level that sort of give not really ultimatum but just to encourage people to mitigate risk you must have incident response capabilities.  
    There are a couple of questions that are still not answered and this is where we would like to welcome everyone's input and encourage people who you know can provide input for this type of discussion.  So there are questions on is there a level of ambition, roughly in balance between regions which are still trying to find that out.  Are there activities noticeable in all regions, continents?  I think the description they are not at the same level in terms of maturity and coverage and things like that.  Do people have access to best practices and guidelines in the same way?  And if not, what are the challenges and can representatives from regions financially afford to participate in meetings?  And can we say one of the requirements of exchanging information or sharing threat information is the trust factor where you need to meet face to face for the first time?  These are some of the other questions that we try to elect from participants.  And if you have inputs in those please feel free to provide them.  
I am going to move on in the interest of time and I think we have discussed some of this earlier.  And we also looked at services and regional collaboration provided by CERT and CSIRTs.  So we have already mentioned that there are a number of platforms at the global and regional and national level and also physical meeting and important for people to get acquainted and establish trust for future cooperation.  And there was an interesting discussion on the cost of CERTs.  If I want to set this up, capability, what are the costs associated with it.  Funding and resources are very important.  Some of the work is quite technical.  
You expect people to undergo certain training and there are tools that may cost money.  In the end people said the cost was very obvious from region to region and even from organisations depending on the constituency that you are serving.  Receiving, processing incident at the national level for the whole population and maybe another organisation who is just serving maybe 200 or 2,000 employees.  So, of course, the costs will vary, but there are some common items that should be considered before you set up this capability.  So things like salary for employee, equipment, tools and traveling to conferences.  Have a look at the document to see what people say are important about cost.
   >> AUDIENCE:  I have a question.  I was wondering if the costs are made public anywhere.  
   >> ADLI WAHID:  It is hard to say.  But I think maybe in some national CSIRTs they probably provide new information, how much funding they receive from the Government.  So some of this information are probably public information, but if it is a private CSIRT operating in a company then maybe it is difficult.  Yeah.  All right.  
    There is also some discussion on the legal system, where the legal system will stop CSIRT from collaborating or sharing information from one another.  So in some ways yes, and I think there is some discussion on that later on in some other section.  But basically from, you know, from past experience in many cases sometimes even with some limitation, with some legal system in place it could prohibit people from sharing information or certain type of information that could be very detailed or private.  But having said that, you know, CSIRT, the community have been around for a long time.  And people are sharing frank information, very generic and this is very important to make sure that everybody is safe where they are using the Internet.  
    Now back to you.
   >> MAARTEN VAN HORENBEECK:  Thank you.  And in the interest of time we have time for one contribution from the participants.  Is there anyone who would like to provide a comment on this section?  Yes.  Please go ahead.  
   >> AUDIENCE:  Sorry, but particularly interested by the legal aspects.  And I want to hear a little bit about it.  I know that if it is a CERT created officially by Governments they will maybe if they are private also they have ‑‑ they cooperate together.  But what if it is a private one, and they are cooperating cross‑border, what about protection of privacy?  What are the legal system?  You could be suspect in one country and not in another.  I want just to have an idea.  That's it.  Is there any rules or any best practices here or any, I don't know, body of rules that is adopted already?  And ‑‑ because frankly you are very occupied by these aspects.  We have intention to create a CSIRT and this is one of the things we are thinking about.  
   >> ADLI WAHID:  You will find this discussion in another section as well.  But definitely I think we take note from our own discussion that some of the specific examples are missing in this document.  So while people are saying that there are some restrictions, especially exchanging information depending on the rules of the organisation, depending on the rules of the country, so those things tend to limit sometimes information sharing or who you can share information with, but generally information sharing and threat sharing are taking place at the same time.  
   >> CHRISTINE HOEPERS:  I'd just like to add that most of the information, I think, for example, 100 information in Brazil that share with other people is about what's happening, not about who is doing what.  So this is one point to take, although we have a lot of teams working with ‑‑ working in law enforcement.  For example, we in Brazil, we are a national CERT but we only work with technical capabilities in trying to help the police.  We don't involve ourselves in the investigation.  We need to follow our law no matter what.  We never share information about who was involved and about details, but we have a lot of information that we have to share about what's going on, what ‑‑ what are the new trends.  So I don't think that is a problem and I think just to put ‑‑ I don't know if it is another one.  There was a lot of discussion about, for example, teams that have a legal requirement to give that information to someone if that prevents cooperating with that team.  I would say that's going to happen.  We need to be careful to create the legislation to prevent the team from cooperating with each other.  
   >> MAARTEN VAN HORENBEECK:  The three of us are not necessarily legal experts in any particular area.  I think what I will take away from this is this is a ripe area for future discussion.  And there are some people that contributed to the mailing list that actually have much more expertise in that area.  You already find some topics there, and I will take this away as a topic that is interested to look in to it further.  Patrick, I am sorry, we are going to have to move along because we still have 14 minutes and we are about halfway.  But definitely we can follow up on this discussion offline on the mailing list and in the document itself.  
The next section is on policy measures and private sector initiatives.  And Adli, I can run through that quickly.  We had some discussion on policy and these were some of the findings that made its way in to the document out of the discussion.  First of all, there have been recent polls from Intergovernmental organisations and also national strategy documents that all for the creation of national CSIRT.  In many other cases national CSIRT or CSIRT came to be due to a specific need such as an incident.  And it was determined as a top‑down approach does not always work.  Most important outside of the way that the team was established were trust and solid delivery of service based on commitments that were made by the CSIRT.  When those aren't there, a CSIRT is without meaning and may not and trust of others.  The role of the Government is very delicate.  A bottom‑up approach was generally preferred by the participants over topdown development.  Trust wasn't really settled in legislation or using NDAs but facilitated over time.  And positive roles of Government include removing barriers to information sharing.  
    Christine covered this earlier, so I won't go in to it in too much depth, but we talked about guides to setting up CSIRTs and we determined there was already quite a few of those in existence.  And it wouldn't make much sense for this group to duplicate those efforts.  
Legal and law.  Most acknowledge that there are delicate areas in the execution of tasks of a CSIRT.  CSIRTs should especially work closely together with privacy regulators as they really support similar goals.  Security and privacy are closely linked.  Cooperation can be hampered by legislation.  It is important that there are differences between the perception and legal reality.  This group should invite more relevant stakeholders in the area and solutions may not necessarily be new laws but better agreements and methods of engagement.  
And then finally we also briefly covered surveillance and Net Neutrality and these were findings of the group.  Surveillance can be an impediment of trust.  Surveillance operations, trust between CSIRTs tends to be lower than usual.  Net neutrality also came up.  Many CSIRTs operate without the need to operate the back inspection.  The back inspection tend to be relatively limited, the CSIRT that provide service to a single organisation such as an enterprise and monitoring services follow applicable law and adhere to privacy expectations.  We have time for one comment on this section.  Anyone who wishes to contribute anything?  Yes, to the gentleman in the back.  
   >> ALI YILMAZ KUMU:  Hi, everyone.  This is Ali Yilmaz Kumu from Deloitte Turkey.  And I chair a Working Group on the National Cyber Incident Response Working Group formed by the ICT of Turkey.  I want to contribute that your point on the perception at the legal reality is very valid.  I want to give an example from Turkey.  The banking regulation in Turkey actually prohibits sending of banking related information abroad and needs to be stored and processed within the geographic boundaries of Turkey.  However there is a perception within the banking industry that also includes any information, security related information such as getting any service from CSIRT abroad or even within the country that is outside of their own jurisdiction.  So even though the legal reality doesn't really mandate that there is a perception that they have to work within their own resources which oftentimes results in a bank having a very poor level of service when it comes to providing CSIRT or not at all.  
   >> MAARTEN VAN HORENBEECK:  Excellent comment.  Thank you very much for contributing.  Now we will move in to the next section, what worked well.    
   >> ADLI WAHID:  Moving on to the next section, so we have a section where we asked the people who participated in the discussion on what worked well.  So that this could be in the best practice.  A few comments there.  Much are obvious.  So CSIRTs have been working well together and sharing information for many, many years.  Much of this is due to the fact that more CSIRT need to respond to physical threats.  They share information how to respond to malware and how to respond to identity theft and so on and so forth.  Capability building is another area that has attention to make sure that CSIRT respond to incidents.  Capacity building is important.  And there is a global benefit that this is achieved.  
If everyone has the right skills to deal with incidents we could make the world a better place so to speak.  And by everyone believed that there is a lot of things that could be done to improve further, reaching out to further pockets of the society or region or community.  Banking might be very advanced but other sectors probably need a lot of help in this domain.  There are many ways to establish CSIRTs, but a Consensus there is a common logical step for establishing CSIRT.  I don't want to go in to this.  There are five things that you must consider before establishing a CSIRT so your CSIRT could be successful in delivering what is meant to do in the beginning.  
    And everybody also agreed that because this document when it was being made most of the participants were from the technical CSIRT community.  So maybe there are things that are overlooked.  And everyone agreed that we need to present this to the multi‑stakeholder community so that other requirements for establishing and ensuring this exercise of CSIRTs are also being included in this particular document.  
    So I only have that for this particular section.  If there is any input for this I would like to welcome you.  Yes.  We move on.  There is one.  Yes.  Yes.  
   >> AUDIENCE:  We have a remote participant and his name is Benjamin and he is asking something.  Is it appropriate for private sector like the banking sector to be instrumental or influential to create cybersecurity policies in most cases to hold legal framework to mitigate their financial risks and should they be involved in international CSIRT?  
   >> MAARTEN VAN HORENBEECK:  So it is a great question.  The topic of creating the cybersecurity policies I think it is slightly out of scope for this session.  It is a great question.  But I don't think it is really within the scope of this document to try and address it.  Regarding the international CSIRT community, there are actually many examples of financial institutions but also other members of industry to participate in the global CSIRT community.  For instance, some of the organisations we mentioned earlier have very wide participation from both Government industry, academia and even some Civil Society.  And I think that's something that should absolutely be encouraged.  Because in the end when it comes to responding to incidents we are all in this together and we need to find ways to make sure that we can find partners that can help us deal with an incident.  So I hope that addresses the question that Benjamin posed.  
    We have time for one more comment.  Actually ‑‑ okay.  Jordana, yes, you go ahead first.
   >> JORDANA SIEGEL:  Just a quick comment.  It was to add on to what you mentioned from the last question and the remote participant.  Example of what I think, perhaps part of what the question was addressing was how can CSIRTs in the financial sector work with other CSIRTs, and do they have a role.  And I would just say that I think that they have a very significant role which Maarten just mentioned as well, but from our own experience dealing with DOS attacks against our financial institutions in the United States over the past couple of years we have worked very closely with the financial services sector and the information sharing analysis sector and sectors that are comprised of like organisations from the banks and that cooperation with the partners in the private sector was critical to dealing with those events.  So just to say that there is a very important role that goes across sectors in dealing with responding to various incidents.  
   >> MAARTEN VAN HORENBEECK:  Thank you.  And I recognize you had a ‑‑ is it a question or an additional ‑‑
   >> AUDIENCE:  It is a question.  I want to understand what is meant by reflect multi‑ stakeholder requirements for establishing and intruding access.  What is concerned by the multi‑stakeholder environment here?  
   >> ADLI WAHID:  When you prepare a document we prepared it in a short period of time.  And most of the contributors are from the technical communities.  There is a general Consensus within discussion that we should get more input from others, others in sectors.  
   >> MAARTEN VAN HORENBEECK:  I recognize you still have a comment, but we will have to move on.  We can continue the discussion one‑on‑one afterwards on the mailing list or to the comment system.  You also have the next section, Adli.  Yes.  
   >> ADLI WAHID:  The next section is unintended consequences.  Some of these are repetitive of we discussed previously.  The document has more details on.  We said earlier that topdown approaches from Government on creating CSIRT does not always work well.  It is always about trust.  Consensus at the political level, sometimes the work of CSIRT are not fully understood and thus creating policies that inhibit the function of a CSIRT.  The reporting path of a CSIRT may also hamper how information cooperate with them.  Sometimes in meetings people do discuss on how to ‑‑ your team or your organisation handle incidents and whenever we share information with you you need to escalate this to another party.  And then maybe she say okay, from now on I will have to be careful and not share so much information with you.  
Cooperation and development of CSIRTs in different parts of the Government is an area where the group sees potential for future multi‑stakeholder discussion.  So again as we said earlier, you know, the CSIRT people believe that different sectors should have CSIRT capabilities.  Part of it is because maybe sometimes the threats are unique for that particular sector.  There are concerns about how laws around data or data privacy might impact the work of CSIRTs, particularly for information sharing.  You said this earlier as well and another ripe area for a multi‑stakeholder discussion is the impact of privacy legislations on the work of CSIRTs.  So we didn't get that much input.  And there were not many expertise or experts in this area in our group.  And I think that's basically it for my section.  Do we have any contribution?  
   >> AUDIENCE:  Two seconds.  I think what we are saying there are two projects that I was saying to Jordana within the EU that are looking at legal section and we just had a large panel session downstairs.  The point that I would make here that the legal paradigm, I think it is highly unlikely that you are going to see a special legal paradigm specifically for CSIRTs.  This seems to be submitting in to those that are the intelligence community and national security space and those who are not.  And we can have debates around the edge of that.  But that's where ‑‑ I say we in an international sense.  So what I would recommend is that all this dialogue really does need to understand what's happening in the legal communities and this was a group of lawyers downstairs.  
   >> MAARTEN VAN HORENBEECK:  Excellent feedback.  Thank you.  
   >> AUDIENCE:  Can I just follow up on really quickly?  I wanted to know what the names of those projects were.
   >> AUDIENCE:  Network security platform and the EU project mapping which is mapping the Internet.  
   >> MAARTEN VAN HORENBEECK:  Thank you.  We will make sure that those are also reflected in the comments.
   >> AUDIENCE:  I have a contact for both.  
   >> MAARTEN VAN HORENBEECK:  The gentleman at the back.
   >> AUDIENCE:  Hi.  My name is Nicoli.  I am from the College of Europe.  When we started the discussion, the discussion clearly mentioned at the moment there is very little notion of hierarchy between different CSIRTs.  And my question is following up on the distinction between those working in the intelligence community and those working in other communities.  And with your comment there you said earlier were ANISA's report were stating from 50 CSIRTs to about 200 plus today.  What we arrive at a need where we will need a hierarchy to allow for full cooperation or is this something ‑‑ do we actually think that the trust system can work with entities, with a number of entities that keeps growing every day?  
   >> CHRISTINE HOEPERS:  I think this is a good question.  I think it comes from the mindset.  We need to take in to account that Internet is not a hierarchy.  We are a Network of Networks.  We are part of this ecosystem.  We are not enforcing and it has been working in trust because you cannot just like say I have to report.  Some of the things that creation retention and what we said before about regional specificity we try to enforce that you have to report why.  You cannot really ‑‑ it is not working.  These are some things that inhibiting people sharing information because they don't know where that information is.  And there is a lot of work at the end that you need to get the metrics better around.  You need to respond to incidents and you will resort to who are helping.  It goes back to what market said what works well.  People are saying what works well and you are a CSIRT that is effective if you have good services.  And then people perceive that you are helping them to get better and then it doesn't matter much if you are a not‑for‑profit because it will all depend on like trust and especially across borders it is all about trust.  I think it sums up.  
   >> MAARTEN VAN HORENBEECK:  Great.  We will move in to the final section and then at the end we will open it up for final comments.  That section is unresolved issues, insights that we gain and next steps.  First of all, unresolved issues.  We found there is enough room for further discussion on what is a national CSIRT and what is really a CSIRT of last resort.  What does it mean, where does it work, where does it work less.  Second we found that we really need to open up this document to some of the other stakeholder communities even more than just in this mailing list because we saw a lot of participation from the technical community, a little bit less from some of the other communities.  And we hope this meeting can kickstart that process.  And we found that the cost of participation for Developing Countries in the CSIRT community and the international processes that exist is very interesting for future debate.  
What are the priorities there and how can the type of work be funded, and then there are organisations that only work for their own constituency and organisations that assist their customers or actually provide CSIRT type services for work which is something that isn't well covered in the document today.  And then finally is there a role for CSIRT teams in addressing the root causes of incidents?  One of the participants mentioned earlier that their CERT actually does that type of work.  Is that something where there is more discussion possible, for instance, with other communities like law enforcements?  Then some of the insights that we gained, first of all, that the term CSIRT was preferred others and there are other CERTs and there is a legitimate need for them to work together, accept information from others to be able to deliver on their core goals.  That cooperation really is routed in trust rather than legislation.  That success of a CSIRT is tied to correctly determining the deliverables and making sure that there is a good perception of what those deliverables are with the other stakeholders, that there are already many best practices out there which are freely shared and result out of those existing collaborations.  And that there is a need for some type of CSIRT of last resort.  Team can take some ownership if no other parties are able to take it.  
And finally this is where I would like to open it up to you.  What are some things that we can do to have better multi‑stakeholder dialogue?  We need to find mechanisms and we could do that by inviting those stakeholders to the meetings of international CSIRTs, but maybe it is more cost effective to share the knowledge that in the CSIRT community at some other events.  And this is the end of what we have done so far.  But we really want you to provide input on what happens next.  We are technically over time.  Feel free to leave when you see fit.  But I invite you to take the time to share your thoughts.  And I go to the next slide which has the link to the review page where you can provide review and comment if you were not able to do so today.  The floor is open for any other contributions from the participants.  Yes, Patrick.  
   >> PATRICK CURRAN:  I was going to say you have asked for comments and information.  I think that the detail of the operational experience between CSIRTs is one set of dialogues which is really rich.  There is a whole other set of discussions involving other actors which is to do with enabling the wider developments.  And it is everything from the legal to the information.  One of the things we are discussing in the EU is about to do a survey at PKI Federation at level 3 or higher because in policies that we have written so far within the Working Groups is that you can't share information systematically unless you are operating at a PKI level 3 in.  We are going to nations to ask what their maturity is because it affects law enforcement intraoperability and supply chains and so on.  I would offer to you that you may want to think about which groups you want to bring together for the kinds of things that you want to cover.  So first would be excellent for some of the tactical stuff but maybe not so good for some of the strategic stuff maybe.  
   >> MAARTEN VAN HORENBEECK:  Good input.  Thank you.  Yes.  
   >> AUDIENCE:  Excuse me.  I wonder if we can see the creation of CSIRT because the services provide as a future investment because actually the cybersecurity is becoming concern of everyone.  And it is becoming the bus.  We do need cybersecurity wherever we are  and whatever sector.  So if ‑‑ it was to be seen as the future as a new industry.  Who is going to provide the basics?  What is going to provide the help to do something that is professional and thoughtful?  
   >> MAARTEN VAN HORENBEECK:  Thank you.  That's very good feedback.  And there is a lot of information that is already available from the current document.  And also in particular there is a link there which has a link to all the external resources that exist from the various organisations.  Yes.  So on that site you can go to a link that says other resources and it will have a lot of the information that's already available from many of the organisations that was mentioned.  And I think what I am hearing in your question is that it would be helpful to make sure some ‑‑ that is put together and available in an easy way for people that actually want to.  Thank you very much.  Any other comments from the participants?  
   >> AUDIENCE:  If not allow me we are looking for a model that is working.  If you have one, please go ahead and tell us about it.  
   >> MAARTEN VAN HORENBEECK:  Thank you.  Okay.  Case studies might be a very good idea of how people have actually done it.  Would case study help address that?  Okay.  Great.  Thank you.  Okay.  One final time are there any further comments?  In that case I would like to thank you all very much for participating today.  I realise we were very stretched for time and that limited the opportunities for interaction, but please use these resources to provide your comments, provide your feedback.  And right now it looks like the document review period ends tomorrow.  That having been said we are working with the IGF to get an extension on that.  Hopefully there will be an opportunity for at least a few more days.  I would recommend now try to get your input by tomorrow.  If that doesn't work, Christine was mentioning the mailing list will remain open and hopefully can stay a resource for the future to have these types of discussions.  Thanks again very much.  And I hope you have a very exciting IGF.  
   (Applause.)