RIPE 93
2 pm
Tuesday, 27th October 2016.
Main hall.
Plenary
JAN ZORZ: All right. Please take your seats, switch off your phones and all the ringing. All right we are beginning this session. First we have announcement, this slip of paper, it says that PC nominations close at 3.30, that's 15:30, after our session.
If you are interested in becoming a member of the Programme Committee, please take a look at the web page and submit your nomination.
So you have time if you want to do the work and selecting the programme you have time until the end of this session.
With that, our first presenter is Carlos from ICANN. He will be talking about testing an upstream intervention of the impact of category one safeguards of internet abuse.
CARLOS GANAN: Thank you, yeah, my name is Carlos, I am from ICANN and what I am going to be talking about today, I am going to research project regarding the impact of category one safe orders on DNS abuse, I will be basically describing this in a bit.
So basically this is work together that we have been doing at the SSR research team, which is basically four of us. And sometimes there is quite come confusion and how the office of ICANN is divided, so I sit within the SSR research, SSR is security, stability and resilience and we do research, we have another research team, right, so those are folks that are mostly engaged also with IETF and things related more to protocol based while our research is for focused on scientific methods and we also have technical engagement which I think Ulrich is also here and OCTO operations which are the folks that deal mostly with the day to day issues of that L‑Root.
So jumping now to the topic that I want to discuss before I can go into what is the impact of safeguards into the DNS abuse, we have to understand what is a safeguard. For the purpose of this kind of research, we define safeguards as technical operational or contractual measure which is is intended to reduce the probability likelihood of harmful activity.
This is important to understand because it's not only a technical measure, it can be also contractual, right, and this would come to basically to be interesting once I define what is category one but here what we have in the table is basically different examples of safeguards, for the past I guess one or two decades, we have been putting all kinds of safeguards into all kinds of processes, here is an example of a few of those regulations like NIS2, article 28 focus on registration data verification which affects several parties, but there are also other regulations which also define all kinds of safeguards and you look at GDPR, they have all kinds of safeguards there, here I mention risk based protection but another type of safeguard would be cryptography, right, they are kinds of safeguards that are set to reduce the likelihood of things going bad.
But do we know whether these safeguards actually work. As I was saying, we have been putting so they will be on Commission and also in other entities right, we have been putting more and more safeguards into place, some of them with defined identifies tease and some of them with other purposes but the evidence to support whether these safeguards actually work as intended is mostly not there. So here I just have the examples of the article 28 where it would basically say that you have to provide accurate and complete domain registration data but whether this particular safeguard leads to less abuse is kind of an unknown, there's a question we set up in this study was mostly to understand to what extent registration safeguards actually correspond or lead to different abuse outcomes that we can actually measure.
In particular when it comes to category one, these are specific safeguards that were set up ten years ago around 2016, this came into place when the new GTLDs were created and the public interest committee set a set of recommendations so there would be certain safeguards for certain new gTLDs, when you what's important to understand for the presentation is that basically this safeguards are cumulative, right, so from safeguard one to safeguard 10, you keep accumulating them, what we did in the study is basically bundle them into three different levels, level 3, 8 and 9 and 10 but you have to understand the level 8 includes the level 3 safeguards. And level 9 and 10 includes level 8 safeguards, this is increasing. Intuitively if there are more safeguards, one would expect to have less abuse, right.
So if we look at level 3, what we have is three safeguards, one is kind of trivial, if you look at the safeguard one, which is basically you will have to abide to the law, right, so basically that's a safeguard that it basically would apply to anything that you do, the second one is that the registrant has to give notice at registration to contact them and the third one is basically if you are working in.health or.finance, you should also take care of the data from the users of that TLD.
When we move to level 8 on top of these three safeguards, it's particularly interesting to actually look at safeguard 6, 7 and 8 because those are related to identifying who is actually registering those domains, if you look at safeguard 6, what it requires is like the registrants have to have certain licences and credentials, right, so what it means is if you want to operate a dot pharmacy domain, you have to have the licence to actually be able to register a dot pharmacy domain, right, so this is particularly interesting because cat one, there are a lots of safeguards coming from different type of regulations which are related to identifying who is your customer, right, a typical KYC.
One would expect that you would create fiction, right, if you are forcing every single registrant to actually demonstrate that they have certain licences to, you would I a psalm the attackers would have friction and you would lead to less abuse, if we look at the level 9 and 10 special cases, level 9 is basically about avoiding cyber bullying and harassment, it's basically because you have those TLDs that I am not going to pronounce on camera but you have some interesting gTLDs there that could lead to harassment and bullying.
And level 10 basically is about military affiliations, right.
So once now we have understanding that we have three different levels of safeguards, is also important to understand that not all safeguards are meant to reduce the harm at different parts of the attack. Right, if you look at the typical lie cycle of an attack, first you start with the attempted and you might or not send the attack, right and once you are able to enter, when you would like to scale it upright and once you scale it up, then there are some safeguards that would aim at reducing how long the threat stays in, right, so if you think about fish phishing attack, first you would have an attempt, you would like to scale your phishing campaign and once you are able to scale the phishing campaign, then it comes down to how long you run that campaign, in this sense we are trying to answer these two first research questions there, does abuse appear at all in this category one GTLDs and if it it appears, how far does it scale, in this study we did not measure the response, we leave it out, that would be another concept of these safeguards.
So, moving to the method, how did we do it, how can we measure reliably whether safeguards actually work.
So of course the first step is collect data, so we use ICANN domain which is basically a collection of five known block lists or intelligence and these define what ICANN DNS abuse definition /SEUS which is basically phishing, malware and botnet C and C, for three years to the end of 2025, we collected data on maliciously registered domains that related to phishing malware and bonnet controls. Beyond threating the data, we collected zone files and registration data, why did we collect the zone files and registration data, to be able to categorise the different types of registries that operate this new GTLDs, right, we need to know how big the registries are and which are operating within these registries and finally because there is some overlap between the registries that operate in these highly regulated gTLDs with other agreements, we also collected the base registry agreements which are public. When we had this data, we basically immersed it, we had to filter compromised domain names, these safeguards are not meant to act on compromised domains, they are meant to act on maliciously registered domains, we have to filter that; you can read the papers how to classify compromised domains but that's what we did and then we also threw away domain names from ccTLDs and layer ccTLDs because the safeguards were not meant to apply, the total in this is 11 06 new gTLDs we we tracked over a period of three years.
Here is where it comes interesting, right, so if you just look at the times series here of category one TLDs versus none category one TLDs and these are basically the rate of reported abuse domains in the zones, it's hard to say whether there is less or more abuse.
Right. So if you just look at the ten series in some months, category one is above non‑category one gTLDs, in some other months it's below.
So we only looking at the abuse rates there is analysis is not enough to allow us conclude whether the safeguards actually work or not.
So the typical operations of several registries were look at abuse counts and rates, this is not enough, right, and why is it not enough, here I put in three different resources for which it's not enough to compare the basically the ROA rates, basically if you look at the median number of the domains, category one is above 5,000, while non‑category one is around 100.
So just comparing the rates without taking into account the size of the gTLDs will lead to basically misleading inference.
Also as I was mentioning before there are different rates of community and brand TLDs, community and brand TLDs, they have other types of requirements so we also need to account for the safeguards that are operating within TLDs that are multiple requirements.
And third, we also have to take into account the time, right, abuse rate change over time, they can go down, go up, depending on phishing campaigns and detection capabilities. We want to separate whether our disablers actually work or not.
We need a saturation model that's able to capture all of these with certain properties, I don't have time to go into the statistics, just to give you an idea of how the model works, it's a hurdle model and you can see in the picture that basically there are two hurdles, the first is to get into the TLD, the second hurdle is basically how much can you scale the abuse.
So these allow us to answer two different questions. First does gTLD have a reported domain at all and the second is like OK, for those gTLDs that have a reported abuse domain, how big is the abuse.
Again, without going into the specifics of how statistics work by combining these two different parts of the model, the hurdle model, is to have the net let's say abuse.
OK. So now that we understand what I was trying to understand and how, let me move to the interesting part which are basically the results. Do safeguards actually reduce DNS abuse. This is basically why ‑‑ the idea of introducing safeguards is because we want to reduce harm.
And here is the first striking surprising results. Category one gTLDs are actually associated with higher abuse occurrence.
Right. What does this mean. We have all kinds of safeguards but attackers are still finding interesting to actually registered domain under those GTLDs. So if you look there at the pie chart, you can see is the amount of months with at least one abuse domains, it's way higher in category one TLDs versus none category one gTLDs, when we model with the hurdle model, the first hurdle, the results said that new gTLDs and their category one, they have 3.1 percentage points more probability of having a single maliciously registered domain.
Does it mean that then the safeguards don't work. Because you would... actually more compromised than the others.
So this is accomplished and that's why we divided the model in two, when we look actually whether the abuse scales, what we see is that actually category one abuse is scales way smaller. So there is 38% lower expected abuse scale under category one.
So, the answer to the first question that we had is like yes, abuse occurs in this safeguarded TLDs, but once the abuse has occurred, the scale is way smaller.
So the net effect is what you can see here, what we can see is like in total, there is 37% fewer maliciously registered domains per gTLD month. So the safeguard, when it comes to the occurrence, they actually do not work that well for that, but when it comes to the net value, we still ‑‑ the safeguards do manage to contain the level of abuse.
So going back to my initial slides where I defined the different levels of safeguards. You remember I said they were community, level has 8 has way more than level 3, so when we compare them, this is more what we expect, right.
So level 8 gTLDs are 50% lower occurring auths, that means that 50%, that means basically safeguards of level 8 are working to reduce DNS abuse way more than the safeguards of level three.
The interesting pattern here to look at is whether over time the safeguards go up, right, because now we have seen OK, the safeguards work, to contain abuse but is the effectually stable over time.
So, what we can see here in here is like the three years that we monitor and while occurrence kind of stabilises, what we see with the net value is that it highly depends per year. So the first year we got a nice result, we were able to find of reduce the scale by 50%. Here it is it hail has an impact, it's reducing the scale of DNS abuse by 7% but it kinds of increased from 2023 to 2024.
Why is that. This is hard to say. There is the whole phishing campaigns that affected also have year, that could be a reason but what we can see in 2025 the safeguards kind of had even a higher impact than they have in 2023.
So, whether the effect on containment is as stable, basically the answer is not. Depending on the year and depending on the campaigns that the different gTLDs are suffering, the amount of abuse and how the safeguards are able to contain it is quite different.
The other aspect we look at is OK, we have like 10 safeguards, we know now that they don't work that well for occurrence but they are able to contain it. What happens with the different threats, do they work equally good for phishing, do they work equally good formal wear, what happens with malware command controls.
So, what our results actually show is that for phishing, the safeguards actually work the best. More malware, they still work, not as good as with phishing and it's only for botnet command and controls, where the impact was not statistically significant. This was also driven by the data that we have, the fact we did not that have many malicious registered domains, this is basically just the way the population looks like, we have many for phishing domains than we have botnet command and controls but basically, it doesn't work as well for command and controls than they do for phishing.
So, let me move, basically, to the conclusions and takeaways. So for the past I would say two decades, we have been defining all kinds of safeguards. However we have very little evidence whether these safeguards work and how do they work.
In this study, we have focused only on a specific set of safeguards that are related to basically ICANN mandates, for ICANN, it's interesting to know whether we need to introduce new safeguard or keep going with the safeguards that already exist but the main conclusion of the study is the safeguards society for this category one new gTLDs, they produce containment so they are able to contain the level of the DNS abuse but not able to prevent it.
Actually when it comes to prevention, we observe the opposite, they are higher odds of observing at least one malicious registered domain in this new highly protected new gTLDs, the campaigns are 38% smaller, given that abuse occurs. However we do observe that there is a big difference when you introduce let's say certificates requirements, when you require the registrant to actually prove that they hold identity they need to operate that domain, by comparing level 8 to level 3, we can see there is a difference of 71% less abuse on those safeguarded domains.
Finally, of course and a note of caution. It's really hard to draw conclusions, right, we are basically observing what happened in in study, whether the effect of the DNS abuse containment is given or driven by the safeguards or not, is something that we cannot prove, right.
Ideally we will have a particular registration data that registries might or might not old, all kinds of data that allow us to move to a more inference type of conclusion.
But that's basically it. And I am happy to take now any questions.
JAN ZORZ: Thank you very much.
(APPLAUSE.)
Are there any questions?
AUDIENCE SPEAKER: Jim Reid, DNS guy, and deeply, deeply cynical about all of this stuff.
I think the best thing I can do to minimise and prevent DNS abuse is stop creating TLDs by far it's the best thing you can do. But the question I have is your research is rather interesting and I wonder if you and your colleagues have given any thought to applying the metrics and information you have found to the next round of gTLDs to see for example this TLD that's been proposes has got a high likelihood of creating abuse or generating a spam face or whatever, let's not proceed to delegate that TLD, I think that would be also a helpful thing to do and it's a very nasty question to ask, I realise you are not in a position to answer it, I am going to ask you to answer it anyway.
CARLOS GANAN: Thank you actually for the comments. Regarding the questions that I can actually answer, so this comes actually from my recommendation from 2016, right, the fact that this is when the safeguards were introduced, right. Of course the way that I can works, everything is bottom up, now we are looking at the impact of these safeguards having in minds that we are, well we, there's a new round of gTLDs, and I don't think there is any kind of relationship between the analysis and the new safeguards if any that they are going to be applied because those also come from the community so it's the community that propose the safeguards, what we are doing here is basically measure the extent to which the safeguards we had for ten years actually work or not.
The other comment that we had around, let's say the attacker space, right, so I would tend to agree that the, if you increase the attacker interface, let's say you have more domains, the high probability is that something is going to happen and I think that comes from basically probability theory, the more you have, the more chances that something happens.
JIM REID: I appreciate that, it's not just a question of quantity, you can see from the last round of new gTLDs one or two of the TLDs were highly likely to be problematic but ICANN still went ahead and delegated them, I hope there's more in the way of protected measure you ares in place to prevent a repeat of the previous failures. That doesn't seem to be the case and I am very disappointed that the research that you have done isn't being picked upon and being acted on, it wouldn't make a big difference but it would make some difference and I am very sad it's not happening.
CARLOS GANAN: Thank you for the comment, yeah, yeah.
AUDIENCE SPEAKER: I thank Carlos for his presentation and study. So I didn't see your analysis in the metrics of domain price, I think they are very sensitive to price, if you are an attacker, one of the variables you want to look into is what's the cheaper ‑‑ did you look at that and why did you not look into that.
CARLOS GANAN: This is the typical level that anybody would be the first one to act, right, because when you are thinking about the value of the domain for the attacker, one of the things that comes into the equation is the cost benefit, right, if you pay more for the domain, you would like to have more benefit from whatever attack. We didn't look at this because basically we don't have the data, right, if I would have the data on pricing, it would be the typical founder you would have shall it's hard to have data back in time, I would need data from 2023 to be able to see whether that has any impact.
AUDIENCE SPEAKER: So all right, thanks. About the gTLDs, are they like open registration policy, did you account for if they are like restricted registration policy.
CARLOS GANAN: That's the different levels of the safeguards. Some of them they do and some of them don't. That's the part where I was trying to control for, right, so you have brand TLDs, so that's why I added to that so some of them do have those type of restrictions, yes.
AUDIENCE SPEAKER: Thanks.
JAN ZORZ: We have two minutes left, please, please.
AUDIENCE SPEAKER: I will be quick. Part of what you are calling safeguards realistically boil down to attempts to identify the registrant and while the TLD should be allowed to have its own policies about what they want in terms of who they are registrants are, I think there are two main problems with thinking of them as safeguards and doing this overall, the first is who do you deny getting hold of ID is easy, there's 10,000 of them available for 350 dollars right now, you only going to stop potential legitimate users from having a domain by forcing them all to have government ID and by making registrars and registries check that, bad idea. The second thing is that within this, you are only moving abuse, you are not changing, if you don't allow it on the TLD, it goes to that TLD, if you don't allow it on the domain and DNS level, you move it to the content level, it's a constantly evolving game, all you are doing is pushing the counsel certificates around the world.
CARLOS GANAN: Thank you for the comment, yes, on the second question of displacement, that's known right, it's not only within the TLDs, with cyber crime itself, if one country starts pushing in certain aspects, cyber crime is going to move to the other country and the same happens with the domain name space. When it comes to identity, that's actually a motivation for this study, right because we have been seeing the regulation that is forcing more of these identity checks but there was little evidence on whether it works or not. And this study basically is trying to say it doesn't prevent, right. So this was an attempt to actually quantify whether they work or not. To what extent they do. But your reasoning is proof that this is in line with that.
AUDIENCE SPEAKER: OK.
JAN ZORZ: Thank you very much. We are precisely out of time. Thank you.
(APPLAUSE.)
Next one is Orlando, please, the stage is yours.
ORLANDO MARTINEZ‑DURIVE: Thank you.
OK. So hi. I would like to talk a little bit about elections and how they relate with our neighbours.
So the talk is kind of divided on these three slide shows, first I would like to talk about mobile networks and elections and this question that I have within having for a lot of weeks now of any of these can be actionable, can we do something about it and what is on the way.
Some options of how to move through these obstacles.
So, we have these two towns, they are French and I am going to pronounce the name very bad, sorry in advance, one of the towns is called Alonnes and the other is Coulaines, they are 10 kms apart, close together. In the European parliamentary election in 2019 in France, they both voted differently, within own of the towns in Alonnes,, the far right got 31% of the votes. Why? In Coulaines it got 20% of the vote, making it second place.
What is curious, if you look at the income and unemployment, they look kind of the same, no, the income is 717,000 and 18,000 and unemployment is around 15%.
Why this happen, no? They should have both very similar, there is something here and there. What? Before I get back to that I would like to talk a little bit about all this idea that is inside. As we know, networks sense, we have these events we can monitor and see like cable cuts which are down, censorship, epidemics and so on, they show a different signature across all these different vantage points ‑‑ it's very simple to see a cable cut on IXP, and you can track that.
I would like to introduce another use case here. Trying to understand how people vote and by how much they vote, if there is a kind of preference with practice and the way that people use the network. So I have this questions here because I don't know how these different vantage points like IXPs get a view on elections, I am pretty sure on the last one, where I have been working for the last few years and this is that I want to talk about.
This is possible because as you may know, there is these network architecture and you have the network the antenna we see on the building an there's the core network that is basically everything needed to make the network work to connect the users with antenna and then finally the connect to the internet, inside this core network the operators are deploying, they are able to track traffic demands, connect the users with the apps and before the traffic goes to the internet, they can inject some deep packet inspection and being able to try to understand why that flow from Facebook, going to Instagram, is it going to LinkedIn, you have all this infrastructure that is in place that you can use it to sensor.
Based on that, I have this huge question hey, let's look at this two elections in France and we made this collaboration with orange and we gathered a bunch of data and orange is a very good vantage point because it's one of the main or the main operator in France, it has a huge market share and what we did we started looking at the week before the election during the middle of the night when people are at home.
If you are curious, it looks something like this. It's basically two upsets, the first one has the information of work antennas are located, we need to know where they are auto each municipality and the second is the volume of traffic that ever service was being consumed. And then you can match those two and basically say hey at this time at this antenna, this amount of Facebook going on and this amount of data passing by but this is just meta data, we are not able to see any kind of content, it's just traffic bytes, so it looks like this.
Basically there's an antenna and the antenna has some traffic associated with with it and you can say hey, this is Netflix, this is Netflix in bore doe, it has this different pattern and then the question is based on this is signal, can we say something.
We can. But to be able to say it, we need to do two other steps, basically we need to relate how antennas are placed on the space, relate to that to areas where people are both, we need to overlap and the second step is that apps bedesign, because of the different orders of magnitude, you do consume a lot of gigabytes, WhatsApp consumes less, there's a way to make these comparative and we did that and basically we have this indicator if it's higher than zero, this app is being used more than that area... and if it's less than zero, where was it.
So how the political landscape look in France in 2025. So France usually for these two relations you have a 6 and 7 major parties but I want to focus on things, the ones that make the most news and headlines. On the left we have from a LFI, on the centre we have coalition, that is the centrist party, you might think about Macron, and on the right, we have resolution assembly national, the typical far right party.
You don't see between the years but the landscape remains kind of the same. OK the first maps that you can see there, the one that says A is the percentage of both that res bell national got in Paris and lien and green means less. And then you have next to that Facebook and Paris and Lyon, and green means less, you can see they are together, less vote for someone national, less users of Facebook. On the right hand we have also Instagram and online news, and that is high. So there is a difference between the areas where the sample national got fewer votes and at areas where Instagram and online news are being used a lot. Online news are being used a lot. We made these for several apps and we put together a list for several apps and it may be a little bit complex to read, I will try to make it simple. What we have on top is 2019 and below is 2024, every row a bad thing and every column is an indicator, the first column is income and employment and the right columns they are apps, what this is saying basically is that in areas where national, you had a lot of both, Facebook correlate a lot with that and is being highly used. On the right‑hand side, in areas where for example La France got high in both, it's an area where Facebook is being used less.
What you can get from here is this kind of signature, no, for every party you have a specific combination of apps and users, that can tell you something about that.
If we go back to our example, now we can say hey, actually these two towns are not so different. They were different, they were the same on income and employment but really Facebook is being used only 5% more than the whole of France but Alonnes has 21% less consumption of news, on the right‑hand side, Coulaines used Facebook far less, 27, 26%, and the consumption of online news is super higher, 43%. So these two towns, they are the same on paper but they have different preferences to on consumption, we are going to get back to this one more time.
So the question that I get when I present these things in conference, no, it's like hey, all these populations, you need to do model because then the model is the one that can interpret different values and so on, and we did, we made several models but I want to talk about three of them.
So what I am measuring here on the Y axis is the correlation between the model provision and the... the higher, the... . In orange we have a socio‑economic model. What happens if I try to explain the relationship between them just by looking at income, unemployment, population, age, gender, I will get these. If I look at apps and only look at Instagram, LinkedIn, Spotify, I will get the green bars.
But if I combine both, no, I make a model that combine income and LinkedIn and unemployment and Spotify, we get the blue path that you can see is here is higher than the others.
So this was fascinting. Let's go back to 2024, and what you get is two insights from here. 2024 is better, it's higher, but the improvement we get on this model grows over time because before it was only growing by 9.6% and in 2024, it's 28% higher. The apps has a lot of hidden information that can be used on insights about the way that people vote and if we go back one more time to our previous example and I say hey, what will happen if in this town if I try to... just by looking at income and unemployment, I will get this socio‑economic here, basically saying 31 and 29, they should be both the same,but , as soon as I enter the apps on this analysis, you see the result change and now they match the reality, and that reality was that in Coulaines the Alonnes votes was 20% and we are getting 21.
So the different ways apps is these are consumed is important. If you are to want to more about this, we have this paper at IM C, this is the title. You go and check it out and there we like to move to the second part of the talk but before, I need to drink a little bit of water.
OK. So we make the paper, my boss is happy, the paper was accepted. But then I have all this question with me, no, hey this is a single MNO, this is 31% of the country, can you scale this, can you say about this Spain, can you say something about elections and the own answer is I don't know, I would like to know. For this second half I will try to walk you through my way of thinking about this, no, and if there is something we can do together.
So, if we go back to our previous example, I have all these question mark at the end, no, and I would like to unravel what this question might mean, I believe if we look at the data we used for MNOs, we get a preference because we have access to the way that people use the correctly phone and which app they use and so on. But this is data if we get access to in‑house wi‑fi or access to an IXP that's about to understand the consumption in your neighbourhood and all this data is complimentary to each other, they are great for valuation and also great to give additional information on top of that. But we can make this more complex, we can say hey, if I have data around IXP, around traffic volumes, no, for the specific levels, I can have a look at all traffic shifts, does something happen between this election and the week before? Is this pattern the same that I was suspecting? Is there any kind of attack happening and now we are making this bigger bigger picture that's giving you a different vantage point and you can go deeper and say hey if I have data about CDN and DNS, I can have the sense of how often people go to a political website, how often people check out news, and if there is some kind of delegation of service attack happening during the week or the weeks before around the election. And alternatively there is this layer of consent provider, Meta, Instagram, Google and so on, where they have some tools for transparency, where you can go there and check out which is the content that they are putting on their ads, if they are sending misinformation campaign passing by so if you integrate all these different flavours, all these different vantage points, you can get a nicer picture of an election and I would like to talk about these last two.
So in the case of CloudFlare, it's interesting because Cloudflare has this platform called Cloudflare Caida and they are able to monitor traffic because they are in the middle of one side, the user and certainly in the middle, in the case of France they were able to say the national traffic goes down but the amount of why is to DNS for news and the domain increased by a lot during the debate of Trump and Harris in the US, although the traffic went down in a different manner from different states, some states were more affected than others and in the case of Moldova in 2025, they were able to track several service attacks. So I found all this data, it's great, it gets your attention and conflict but doesn't tell you a lot about the way that people vote, it doesn't tell you which party to support, it will tell you something and that's always nice.
Then on the content platform, we have these kind of minutes relationship. For example, Twitter has this API, you are able to access a researcher and get data but that API was closed after it became X and now you need to pay 42,000 a month, nobody is going to pay that per month. And something similar happened with crowd tangle, it was a platform for Meta, you can also get insights about the way that people use all the platforms and that platform was also shut down. And then you find what's happening on the API of TikTok.
So the question that I have been having is like do we need that? Can we make something ourselves based on the infrastructure that we already have, based on the infrastructure that we already measure? And I already see good things. We already have passive sensors, we don't need platform API, we can make it aggregated so we can identify users and this will allow us to track for a long period of time how our democratic process changed. Over time.
So, this is the part where everybody becomes a little bit fussy in my head, I don't have answers, only have questions, this is why I am here. So think that we can make election observe tree and it's hosted by a neutral entity, OK, on the upside, we can have producing this data that might be helpful like correlations, models about the way that people votes, also monitored for participation, can do a spatial analysis and tell you which areas are more likely to go, to vote, and also you can have a nice set of protests, epidemics, digital divide, all this can be measured with the data and the website here, we need a contributor, the contributor in my mind is basically MNOs, IXPs and everything and it's a little bit naive to think you can ask them for data and they are going to say yes. You need to give something in exchanges.
I want to talk more about that in a second. Who might care about this, no. Why you should think about this in any case. I think election monitors, regulators, measurement community, researchers, political researchers and network operators, they might get all this. OK, so let me go into a complexity here.
So every time I talk about this, I am pretty well aware that this is not straightforward at all, there is a lot of open problems, I have only have some suggestions but this is why I came to RIPE, this is a typical question that I get and I would try to analyse those questions with you.
OK, you want to make this ‑‑ who is going to own the observe tree, it's not going to be me, I cannot but I think a non‑profit can do it, an EU funded or public NGO can be the owner of that, even can be RIPE.
It can be a RIPE working group. It can be a typical European digital media observatory consortium, so there's lot of institutions that can try to own these processes, it's not me, it's someone else.
But what is important is it needs to be someone trusted, neutral and credible. This is one question. The second question is why an operator will share this data, why. What is the intensive, so I only have some ideas. The first idea is that hey, if you share the data with me, I will need to process it and part of that processing, I can give you some insight that makes your operation better, some insights that you can use for your costs, so you are saving money because I can help you to understand your antennas are malfunction, where your network is having a high latency or having an issue.
The second if you go through the data, you can get access to the data and now you are is a begin knell networks you are an MNO and you have data about IXP, DNS, CDNs so now you are able to have a more holistic view about all the networks, including your competitors.
The third point is maybe we can just buy the data, if we have enough funding, as simple as hey, forget about all the positive things, I want to buy the data and you give me the data and I will use it for this observe tree.
Then there are these something middle, there is the public good, we are doing something nice, no, and something nice also relates with better political relationship and better PR so the operators can get this, hey, we are contributing to an avoiding misinformation, we are contributing to make the process more transparent and all these non‑negotiables, all these some kind of important points too.
So to my question, and I will finish, the question is what actually leaves the building, you want this data but you need to share. You only need to their relative indicators, I don't need any kind of absolute ‑‑ or something that can compromise the operator, at the same time we can have an aggregation layer that doesn't compromise the user so all this analysis is not based on how a person votes, about it's about this municipality, this town, this province. Last question is, who pays for this, where the money is coming from? The money is important.
So most of the observatories that are today get money from grants, donations or they also try to find a way, what they can sell that they are producing and then they can give extra benefits to a contributors of the data. So
Yeah, these are the kind of questions I get, the kind of answers that I can provide but as you can see there's a lot of complexity, I would love, I would love to talk about this later.
So, going to end, the different ways I want to share as I mentioned MNOs, IXP, ISP, they won't need new hardware, they carry complimentary signals, important for elections. So I will would love to talk about this, it can be any of the topics, research, funding, MNOs, policies and final take away, this is the ACM paper if you want to check it out and there are elections in, one in Spain in November and three elections in 2027 and finally you can reach me at the addresses on screen.
I know what I am saying is not trivial but I don't think it's impossible, it's difficult but not impossible, it can be done and most of the people that can make this possible are here.
It's you. So thank you, this is my talk. And I am happy to take any questions.
(APPLAUSE.)
JAN ZORZ: Thank you. I see at least one person going to the mic.
AUDIENCE SPEAKER: Hello, I'd like to ask a question like the engineering part was interesting but have you considered the humanities aspects of that, whether it is OK to predict elections or use the network data to interfere potentially with elections and who has access to this data versus who doesn't and what they can do with it with access to such systems?
ORLANDO MARTINEZ‑DURIVE: Thank you for the question. So the way I think about this is like this is a knife, you use it to cook or do something bad. This is the part that if you have a robber... on the way that you build this, then you have a knife just for cooking, that is a good thing. So also we can't frame this in such a way that avoiding any kind of harm, we just want to get understanding why people vote, incentivise people to vote, no? So it's just to make the democratic process even better, but it's a matter of how it's implement and this is why I trust that I go to like RIPE, the European Commission can do a better job in that sense.
AUDIENCE SPEAKER: Thank you.
JAN ZORZ: We have an online question, then it's Rudiger and Jim and we are closing the queue.
AUDIENCE SPEAKER: Online question: "Thank you for the presentation, using operators DP I data to infer communities political preferences, seems to cross a sensitive ethical line, even when when the data is aggregated. Are we at risk of normalising the use of communications infrastructure for political surveillance, what safeguards would prevent this approach being used by governments or political parties to monitor or target particular communities?"
ORLANDO MARTINEZ‑DURIVE: Yes, thank you for the question.
It's a tool. Tools can be used in a positive or negative way. I think we have in place a lot of tools in place and the only way that we can control that is by having a trustable, neutral and independent body that handle that tools, we don't give that to anyone. This is the reason why we have ICANN and we have RIPE, no, that we must so I see all the complexities but I think that done in the proper way can outweigh the complexities and make it better.
RUDIGER VOLK: I am also kind of asking for the DP I. You are are you actually assuming that DP I is available on essentially all of the Xs and ISPs, I would be surprised if that assumption is true and yes, I know the mobile operators are used to doing much more expensive processing and sometimes doing things that as a citizen, I am not really happy about.
ORLANDO MARTINEZ‑DURIVE: Thank you for the question. I am not assuming that an IXP is doing DB I, I'm working with the same data that an IXP is already using, that is traffic volumes, a change on the traffic patterns and who is member of the IXP, so we don't need the DB I and I think there is an official that we don't need DP I, one of the questions you will get is how can you validate the results from the... I know a data source from an IXP and the IXP is not doing Dpiand I can trust the data from... is similar.
So in the specific case of DPI, it's only for (inaudible) operators, they are the ones that tend to do this but I don't know for the IXP.
JAN ZORZ: One minute.
JIM REID: Thanks very much, fascinating presentation. Lots of stuff for people to think about here. But I was approaching this from a slightly different direction, I am not interested where you got the data, I am reminds of a favour radio programme when the BBC radio four, everybody check it out, one of the things that comes through time and time again correlation doesn't imply causation and I just Googled an example, did you know correlation between Nicholas cage movies and people drowning in their pools at home. There's no causation there but there's a correlation there. I am very concerned that you are presenting statistics here noting those correlations but are the correlations leading to causation and that's been left unsaid and that might need further work from you.
ORLANDO MARTINEZ‑DURIVE: That is so much, this is a critical question, we get it every time. So it's two answers, I will make it fast. The first ones that it's not clear to me, I don't think it's clear to anyone if it's the way that it's what you can consume what make I vote that way or what make you vote that way relates to what you consume. So there is these relationship there.
That is not clear what drives what, no? Because I support this party, I am going to consume this party or because I consume this content, I am going to support this party.
Then to make everything more complex, on the specific question, this is the reason we make a model, when you make a model and you integrate all the socioeconomic indicators with the apps, you try to get a sense on the causation, no, because now you can say something like hey, Facebook is correlating with income and is income the one that drives the way the people vote, when you put it together in a single model, you say if Facebook is still ... it's because it's actually... so we can talk more about this, there is a bunch of data on the paper but we were particular on not mixing correlation with causation but yeah. Thank you.
JAN ZORZ: Thank you very much.
All right, we have lightning talks now and I believe Clara the members one is online, right. OK. So this is Daria, do we have her online, yes, the floor is yours. You have ten minutes including questions. So manage your time well.
DARIA SHANTALOVA: OK, let me check if I can... OK. Yes, wonderful, all works.
Good afternoon everybody. I',m an engineer at Meta, my name is Daria and I am going to talk you today about route leak incidents that happened in our network. This is a network that connects our data centres. There are incidents naff happened on your network and multi‑layer routing domain protections that we have built at Meta for preventing these kind of issues.
Agenda. So I am going to walk you through the incident that has happened in our network. A route leak and why no filter caught it. I will also cover why you might have these kind of problems and issues and what are the preconditions for them, then I will walk you through our three layers approach to solve this kind of incidents, it will cover BGP communities control, prefix filters and control plane health monitoring.
We will finish with architecture summary of the solution and lessons that we learned during resolving this kind of issues.
And I will finish with how you can apply it without our stack.
This OpenSource tool. So the incident that happened in our network. We have expressed a network that inter connects our data centres and we have internal network domains that are connected to this backbone network, BGP, so what happens, this internal domain has started to advertise to express huge amount of practices, the particular details I would like to know can you say on is all prefixes that we are advertise to backbone network had a valid BGP community attached, all prefixes were through our route maps, once it's propagated through the backbone network, they will advertise and mini routers in these region are not able to handle this amount of prefixes and routing demon has crashed, we ended up with an outrage that was caused by prefixes that had completely valid BGP and filters that allow this prefixes to go through.
So why... as I mentioned, all prefixes had valid BGP communities, it was allowed in our road maps and BGP community based filters asked the question is this BGP community, but there is a gap, it doesn't doesnt answer if this prefix actually carries in BGP community tag and this gap is potential for new route leak incidents that can happen in network.
So we have two problems in this incident in prefix overload and the route leak. The prefix overload can be fixed with limiting the form of the prefix, and the route leak can be fixed with a permit list, which allows which prefix may cross the boundary.
So while one solution is not enough. While in case of prefix overload like we can prevent from maximum amount of prefixes but we can't prevent if there are like a wrong amount of prefixes that can cross the boundary. And in case of prefix limit, we may differ between right and wrong prefixes, but we are not protected against if we have a huge amount of right prefixes crossing our boundary so was solutions unnecessary.
Why this problem may be related to your network: There are several preconditions in case that can be a potential for this kind of route leak incidents, first if you run several routing domains and BGP carries prefixes between them and if BGP communities have actually influence where these prefixes in your network can propagate. We already have well known solution is internet route registry that binds prefix to AS number so we wanted to build something that kind of analogy that binds prefix to a certain BGP community.
So we created a 3 Layer solution approach that kind of covers this incidents from different domain, it is BGP communities, then another layer is prefix filters and the third one is control plane health monitoring.
So BGP communities is creating identity tagging and if it answers the question what is the prefix actually is, identifying the prefix in the network. The frequencies filter acts as permit list and answers the question should this prefix be at this network boundary and the layer control plane helps monitoring, monitors production state and checks that it works according to the real intent, does reality match the intent.
JAN ZORZ: Daria, you have three minutes left.
DARIA SHANTALOVA: BGP ID goes into details about each layer, we have invented all BGP communities that exist in our network and locked them in our BGP communities database. In this database we mapped the BGP communities in ID. What makes a BGP communities ID is it defines where prefix originates from and each frequencies carries exactly one BGP community and you can sync it with the passport and visa. So we give each frequency a community that says like each network domain in this allowed to pop great to.
Then we have a route registry prefix filters layer, it is the equivalent of internet route registry for the backbone, so we locked every prefix that existed in our network and the domain is came from and the community, the BGP community it carries. So in previous slide we were talking about BGP community database, this is the same but for prefixes. And it implemented as a prefix list model, only explicitly the registered frequency overload and it is deployed via a script on every device and allow only prefixes from this route registry.
And the sub layer is a control plane health monitoring. So it's not just to deploy these solutions, it's important to make sure production stays much the same so we have continued validation from our routing state. We continuously collect BGP update from every device and check all prefixes has BGP communities, if we have any unregistered prefixes and in case we have any issues, we alert our own comm.
JAN ZORZ: One minute left.
DARIA SHANTALOVA: Architecture summary. What happens, if prefix comes to to our networks, it faces a prefix list, route registry, if it's in the list, it will pass, if not it will be dropped. There is a second layer, BGP community ID, BGP community route maps also check in, pass or not pass and monitoring, checking R RIB if in inconsistent with that.
So applying this without our stack. We don't OpenSource our solution but you can implement it with OpenSource tools like Netbox for your own route registry or BGP communities database in case of prefix generation, you can use A route servers or bat phish and for BGP RIB collection is E X A BGP collector and thank you very much, I am ready to answer any questions.
JAN ZORZ: Thank you, Daria, you are spot on.
(APPLAUSE.)
We don't have time for questions, Timo you want to ask questions or talk about how unsafe our domains are. The stage is yours.
TIMO HILBRINK: Hello, good afternoon everybody, my name is Timmy Hilbrink, I work for a small ISP in the Netherlands. This presentation is on my personal account, that doesn't really matter. The font I used is call American typewriter, I thought it was appropriate
On the 26th August, the Secretary of State Marco Rubio posted this on social media I believe it was on truth social about a they were going to sanction an infrastructure company in Italy for supporting extremist networks. And they were going to do that with everything they could.
Sure enough that same day there was a statement on the US government website that the office for foreign asset control was designated, Autistici/Inventati as a designated global terrorist. What is this? It's an Italian non‑profit organisation that runs services for political activists, they are non‑profit as I said, and they are in, they are actually run an association complying with all legal regulations, legal responsibilities and also obligations and controls. They run, did run 16,000 mailboxes, 1500 websites, 5500 mail
ROBERT LISTER: S and 10,000 blogs.
Oops. Two days later, their website was gone, not only the website was gone, the whole domain was gone.
So what happened here?
Well look at the ICANN Whois information, we can see that the status of the domain is server hold and was updated on the 28th August. So two days after the announcement came.
Now, server hold means a status code set by a registry operator, your domain is not active in the DNS. So what basically happened is that domain of Autistici was removed from the DNS. How did that happen, with the registry for .org is a public interest registry, PIR.
They are created back in 2002 by the Internet Society to manage the dot Org TLD and they are headquartered in the US.
So suspicion was already pretty early that PIR following US sanctions had placed the domain on server hold basically removing it from the DNS.
About four weeks later Sally Wentworth, the CEO of the Internet Society, released this statement, I didn't include it here completely, you can look it up on the QR code, basically admitting that because they are headquartered in the US, PIR had to comply with the sanctions from the US government and take the domain out of the DNS.
So now everyone with an Org domain is afraid this could happen to me as well. But what about the other TLDs? Dot Org is based in America, run by PIR.
.net is run by Verisign,
.com is run by Verisign, also headquartered in the US which means all these organisations would have to comply with sanctions as we have seen for the Autistici.org, that's a bit scary but there is a lot of internet governance organisations that manage all this and should maybe prevent this from spreading or getting worse or more domains getting blocked.
Yeah. IANA is based in the US. ICANN, ISC, ISOC and therefore the IETF, the only exception here is the IGF which is based in Switzerland, did you know by the way that Switzerland is only one of two countries in the world that has a square flag. Very interesting.
So, this is, I think, something we have to look at. We have something we have to make ourselves more resilient because there is more sanctions on the way. I mean this was just a sanction based on some terrorist claim for an organisation in Italy.
But the US government have in August also announced an operation economic out cast on the Iran, where they will be sanctions lots and lots of organisations in Iran and the companies that supply them with services.
Then we have the sanctions on the international criminal court which is based in the Netherlands where I live. The US government announced they were going to take it down brick by brick and they would do everything to make their life impossible.
This could mean that services that we as a community or members of the community provide to these organisations that could mean that you will be sanged as well, because remember, Autistici was not an active political organisation themselves, they were just a service provider for these organisations.
So my question is: Should we still depend so heavily on gTLDs or other TLDs that are US based if we want to keep our domains in the DNS?
There's some URLs that you can check for more background information.
JAN ZORZ: We have a question. (APPLAUSE.)
AUDIENCE SPEAKER: It's very cruel that this is a lightning talk because I think the Q&A session could go on for hours.
TIMO HILBRINK: We can do it elsewhere.
SPEAKER: Ben from BGP tools or BGP.tools, so I have a stake in this as well. I think the most interesting thing, interesting is not the right word, the more concerning thing that comes to mind is we are all around in RIPE.net and of course the RIPE NCC has, as part of its service region, Iran and what would happen tomorrow if RIPE, something in the great nebulous cloud of the US government decided that they didn't like this and was going to just zap the domain, or zap RIPE in some interesting sanctions? I know that RIPE already has a lot of trouble with dealing with sanctions, I feel like the US sanctioning it it would be an an extremely bad day at the office.
TIMO HILBRINK: This is exactly one of the reasons I brought this up because there is a lot of networks here. Organisations work on the .net or .org TLDs and this could mean really bad things for your organisation and RIPE is only one of them.
AUDIENCE SPEAKER: Sorry for doing a double question, I will throw a small conceptual hand grenade into the crowd: If RIPE did have to move away from .net, what would the TLD have to be, because it's not like we could use .oh you are part of the service region involves not the EU.
TIMO HILBRINK: .NL? .Eh...
OK. Jim.
JIM REID: Tim, thanks for this, it's very thought provoking, I don't think there's any easy answers here. One issue is that you start shuffling around the TLDs you are still not getting further forward, and then you say let's pick a ccTLD and try opt out of any kind of United States government control, we are still back in the same game because ultimately IANA is headquartered in the United States, so it's subject to US legislators whims if they decide to do something stupid as they have been known to do.
TIMO HILBRINK: Absolutely, it's unlikely I think that the US government would take down a whole TLD. It would be very stupid but it is possible.
JIM REID: It is possible.
TIMO HILBRINK: It's something we have to consider.
JAN ZORZ: No, no.
JIM REID: Just to answer the point that was mentioned by my colleague a few, moments ago, let's move to another TLD. Could you imagine how long it would take this community to decide what the other TLD would be!
TIMO HILBRINK: Consensus based, right?
JIM REID: One quick point here, some of us might remember there was some part of the United States government came knocking at the NCC door a few years ago about the fact that the NCC was doing business with Iran, I don't know how it was resolved, I think they asked the Dutch government to have a word with the federal government of the United States, so we have had these situations before and we are going to see them again.
TIMO HILBRINK: In this case the Dutch government wouldn't have any say because it would be a sanction directly from the US so...
JIM REID: Indeed, but even in that situation I think the Dutch government was able to prevail, I don't know for certain.
JAN ZORZ: Jim! We are cutting the line, we are way over time. OK, you can go because you said you are sorry. Please Alex! He was before!
SPEAKER: You focus here a lot on the US, rightly so I think, but the European Union also has been sanctioning people left and right. I don't think that is a viable option here, what would you suggest?
TIMO HILBRINK: Well a very short answer but the European sanctions were not based on the global DNS system, they were enforced on resolvers at ISP level.
AUDIENCE SPEAKER: That's not quite what I mean, it's that the EU has not actually done this specifically but they have been sanctioning people and organisations.
TIMO HILBRINK: Absolutely.
AUDIENCE SPEAKER: And they will get to this point at some point, I don't know.
AUDIENCE SPEAKER: Do you want two words? Cool. If we are going to, if the idea of dropping TLDs are on the table, surely the solution is drop DNSSEC, that's the main mechanism that you could really do this with, right? I also hate DNSSEC, it wouldn't hurt.
JAN ZORZ: Thank you, Timo, thank you very much. Andre.
Please. Ten minutes.
ANDREI ROBACHEVSKY: Thank you for giving me breathing space here. I am from global cyber alliance and this is the organisation that host Americans and acts as a secretariat of MANRS and I will give you a quick update about the work the working group is reviewing on MANRS, the operators programme.
All right
A.
So what is MANRS? It's mutually agreed for noms for routing security and it finds a baseline, it's not a recommend set of requirements, it's a growing community that commits to this baseline and we verify, it's actually available on a website, you can check, verified commitment to the three requirements compliance.
It started 12 years ago by nine operators and now it's 1400. And more and growing. So it's time to look at those requirements because the industry moved along and the technology with it as well.
We did a survey just to say what the appetite is on reviewing those actions and the result was interesting. People said that in general the baseline is fit for purpose, we need nor clarity on requirements that people know where they stand.
And the survey indicated the network operators problem by way the largest problem in MANRS should be the first to get reviewed.
Another piece we introduced last year was a MANRS development process, that sort of was a standard development process that defined the stages, make it transparent and predictable, so we used this process by the way to introduce the process itself and we use it to review the charter and now it's the turn for the network operator's review and that's where we are, a working group has been formed and it has been working since June and that's what I will talk about.
So this is the working group, 22 members. Bi‑weekly meetings, making good progress, if you would like to look at what the work looks like, the minutes are available online, you can is a look.
We bring this to a conclusion, the group last call followed by the community call for comments.
So why I am here, why I am doing this talk, first of all immediate feedback, I don't expect a line of questions and answers because of time but please stop me in the hallway, there are a couple of members here in the room as well, stop us, provide any feedback, that's very important to us.
There will be a last call and community last call and why it's important because we want to this requirement represent not just a small group consensus or a MANRS participant consensus but more reflects the global consensus in the community. If you are interested, if you can contribute to that, please subscribe to this mailing list and participate in the last call discussions.
So what are the main things? Well we split the document, we decided to split this into two to make it more clear, one is requirements and how those requirements, the auditing stuff, we made it clear by using IETF queue cue words, "must", "must not", so the meaning is not ambiguous. Obviously RPKI made a long way since 2014 when it was, when the MANRS was launched, so ROV is a must. IRR is a should, I will explain why is that.
Another requirement enforced, first hop Ann, the ASN you peer with, the neighbour is the ASN the path you receive and bogon, it was very complex, now we limit it to special purpose registries in IANA and well ASSet segment, it's probably good to include in this set of requirementsm so ASSet shouldn't be originally ‑‑ it will break our review anyway, but that's a requirement. So the main challenge in this action, we sort of discussed is the legacy resource holders in ARIN who do not have access to RPKI. And fall back to IRR for large operators which might require huge filter list, hard to maintain.
Action to 2 and Action 3 didn't sort of undergo substantive changes, for action 4 we had a lot of discussion and I think we agreed there must be 100% coverage of registrations for RPKI so every announcement that participants should announce should have a corresponding valid ROA. Now IRR is still in use so we can't ignore this in some operators are not fully high grated to ROV, they are using IRR, non‑MANRS, other operators, so IRR is still there but there's some challenges and challenges first of all what's the set of IRRs that we define, I will talk a little bit about that.
In the next slide.
One of the challenge we'll also discuss is that with IPv4 shortage at least space for prevalent and in many cases operators have real trouble originating those non‑BGP customer lease prefixes because customers do not create the ROAs or these companies not able to create ROAs but they are responsible for that so how we deal with that.
So to summarise, must be registered in RPKI, I should also be registered in the IRR, we softened this requirement and objects registered in the IRR might be consistent with RPKI, we have had some presentation today so it's a problem, it's good to keep it in sync. ASSet, if it's used, not everybody uses ASSet but if it's used, must have a higher... and then defines the operators customer must be registered in the IRR and PeeringDB and ought number objects must be registered in the IRR.
The real challenge is how you define the IRR and ideally we would like to stick all auth 5 areas for RIR IRRs sorry but some of the committee will be not able to participate for instance Brazil Jan community, NETCONF no access to lack anybody IRR and that would be a problem for them.
So when looking at this we also thought of a few things. First of all MANRS defines a baseline, it's not highest sort of strongest security requirement. It actually geared for for adoption, it says the wider MANRS baseline is adopted the more effective we have rather than just a few operators implement very high standard security requirements so we keep in mind that not everyone that is possible should be in the MANRS baseline.
We need to understand how the changes in the baseline impact the existing participants, right, so but not just numbers. Also adoption path, how visible it is. To get to the come state for members. What we did, we did some analysis, I will not go through this because I have no time but you can look at the slides and we played with different for instance here we looked at different sets of IRRs and how it might affect the existing participants, their compliance level, you see the errors red and blue of how some of the participants will reduce their compliance data, some of them increase their compliance status, the same for ASSet, we looked at PeeringDB and looked at how many of them are higher hated and how many are registered in the areas of IRRs and so on and AutNUM registrations, we played with different scenarios and that's useful for making.
I'm done with this presentation but this is a shameless plug, there is a MANRS steering committee election starting from tomorrow for three days, it's who is eligible to vote, MANRS participants, if there are MANRS participants in this room, please look up your mailing box there should be a mail from the voting system, Big Pulse we use, and please vote, that he very important and that's it.
JAN ZORZ: Thank you very much.
Are there any questions for Andre.
We have four minutes over time.
‑‑ we are four minutes over time, OK, there is a bar outside so you can talk to Andre.
ANDREI ROBACHEVSKY: Stop me in the bar, in the hallway, I am happy to discuss that. Thank you.
(APPLAUSE.)
JAN ZORZ: OK it's a break now. You cannot apply for the Programme Committee any more, the time has elapsed but the social, yeah, we have been told that for the social, the bus leaves at 8.20 and 8.30 from the hotel:
OK. And rate the presentations please.
Thank you.
Coffee break.