the AI Act, even though we've got somewhat of a changed timeline on that now, we'll discuss how
we can prepare for the AI Act and I think more importantly as I mentioned, I think the bigger
picture behind the AI Act being as technology evolves and as the rules change, how can you
prepare for a regulation that is ever evolving as well. So with that I'll let the panel introduce
themselves, starting with Suzanne at the very end. So good afternoon everyone, I'm Suzanne,
I'm an associate professor of technology law, so I teach all related technology law topics including
law and policy of digital services, data protection law and I do a lot of research focusing on the
EU policy including how to properly enforce DSA rules, GDPR rules and so on.
Hello,
everyone, good afternoon, so I'm Bertrand Payès, I'm the head of technology of innovation at
BACNIL which is the French Data Protection Authority and I oversee the activities of the newly created
AI pattern that we created to work on, especially on how GDPR and other legislation apply to AI
systems which are new systems for us as well, even if we have stopped using them for quite a time now.
Thanks, so just a reminder that this is sort of a workshop for us as well,
so there'll be a lot of people who will be joining us, so if you want to come in and talk a little bit more,
we'll be happy to do that. So if you want to come in and talk a little bit more, we'll be happy to do that.
be time at the end for a Q&A and for you guys to ask questions about specific things that you
want to find out from our panelists, specific tips that you'd like to get from them. But I'll get
started right now with my first question that I'll preamble, I think, so when we think about
the things that developers and business leaders and decision makers in this field should be
thinking about now rather than the future legislation, a helpful way to break this down is to think about
what are the laws that affect all sectors and then what are the laws that might affect various
verticals specifically. So if we start about the more general state of things across our models,
what would you think, what would you say are the key blocks, the key themes
that business makers in this space should be thinking about when they are building an AI model?
So yeah, AI is pretty much like any other tool, so basically you
need to look at what you're doing and think about what you're doing and what are the impacts of that
to actually know which are the laws that are applying to you. And so basically there are
already a lot of existing laws that are applying to a lot of different tools. We've been reading a
lot on intellectual property laws applying to AI or how do they apply, but there are also data protection laws,
in Europe there is the famous GDPR, there is also consumer protection laws, competition laws,
and so all of these you know transversal omnibus topics can also apply to your business.
So I think these are like the most various topics.
We were discussing earlier, you can add anti-discrimination laws, which is also
those that are very legal at the same time, not with
big technical approach, but will certainly apply also to many cases where you can have
discriminations and the AI system will be used in that context. So HR, hiring people,
credit scoring, that kind of thing. Great, thanks. And we'll come to the
blocks I suppose one by one. We have experts, specifically experts in data protection here
as well, so I think we might spend a bit more time on this, but before we get to data protection,
could you give us a quick date of things on IP, which is the first block that you mentioned, and
what are the kind of key things to think about in IP?
So maybe I'll start. I think IP is a sector where you have a lot of lobbying influence,
and so that being said, we usually talk about the big tech lobbying, but we don't realize
how much lobbying the IP sector has been doing for the past hundred years, when you look at the
extension of the copyright limit in the seventies, you really can understand their influence.
And so I think there is right now a lot of litigation pending in various jurisdictions,
especially in the US. Last week, I think, a California judge dismissed lawsuits by artists and
representations of artists on, again, notably, IP.
Okay.
Thank you.
Excellent.
Thank you.
mid-journey, where they said that there was no copyright infringement, so they had no
claim, so there's this decision, but it's only one judge in California, but there is
also a lot of other decision pending, and I think we will see a lot of litigation in
the US, maybe a bit less in Europe, but we know there is also a big lobbying by institutions
protecting authors in France, for instance, where you have a bill actually offering to
better protect artists with AI, so there's a lot of uncertainty.
My main takeaway would be this, there is a lot of uncertainty on this field.
So basically, keep an eye out, there is a concrete set of laws already that you can
refer to if you're building in that space.
Unlike data protection, as you mentioned, so we do have the GDPR.
That is a very specific law that you can refer to, and that is there to inform people
in this space.
Would you say that it is basically the key law that you should refer to, and Bertrand,
I know that Gnil has sort of published guidelines around this, so maybe it would be useful if
you run us through basically the key things to keep in mind to navigate the GDPR today
from an AI perspective.
Yes, thank you.
Just to build on what Susan said.
Yes.
For many AI, and when we talk about AI right now, we often think about generative AI.
I think there is typically a difference between copyright material and data protection.
For generative AI, it's obvious that having copyright material allows you to better perform.
It's also true for data protection, but it's not as key as it is to access copyright data.
It's not as easy as it is to access books or actual code from GitHub or something else.
And we saw, I think what we saw in the data GPT investigation from our intelligence counterparts
in the beginning of the year, I was myself surprised to see that data protection issue
rising so quickly, and even more quickly than copyright.
And I think that's a very important thing.
Thank you.
And I think that's a very important thing.
Yes.
And I think that's a very important thing.
Yes.
And I think that's a very important thing.
But I also was convinced, and I think it's probably something I would say, that the
risks at this stage are not that important for generating generative AI because it's not
the goal of, in the case of OpenAI, it's not the goal of OpenAI to target people.
They have some data in their training dataset, but they do not want to release it.
Yes.
That's a very important thing.
That's what I'm trying to advocate.
Yes.
exploit the data directly
for the general purpose
and there can be use case of use
of that general purpose
that is aiming at targeting people
but still it's not like personalized advertising
or social networks
where the key business model of the company
is to target data to actual person.
So that means that it's important
but normally you can manage that in many cases
and that's what we tried to tell in our first guidance
that we published a month ago
where we tried to lift some of the risks
that one may see in GDPR when training AI
so we focused our guidance right now
on the processing of training AI
not deploying it
but just training it
and that's what we did.
And we said a different thing.
We said for general purpose AI
such as NLM for instance
we said that it's not problematic
regarding the purpose limitation principle of GDPR
which is not of use
because the main idea of purpose limitation in GDPR
is to limit the purpose
and it's typically the opposite
of doing a general purpose AI
but we tried to say that the purpose
is a very broad notion in applying GDPR
and what I would say is that
you cannot be compliant with GDPR
if you have your own purpose.
If your purpose is to say
I want to do an AI because I want to do an AI.
This is not the purpose in terms of GDPR
but obviously you can say
I want to do an AI that is aiming at discussing
casually with people about everything.
Which is basically a large language model.
You do not need to be very precise
but you need to have a goal, a purpose
and this is really one key principle of GDPR
that you need to process certain data
only if you have a goal that you are able to
legitimate and determine.
Perhaps the second aspect that
I would cover two other aspects in our guidance
but there have been seven fish, seven papers, guides
in English as well, so you can read that.
And we will extend the contribution period for one month
because it was ending yesterday
but actually you can enter later.
The other aspect is how you cover
data minimization principle
which is also in GDPR
and may also seem contradictory with the fact
that you would need large scale databases to train AI.
What we said and what we see
in the profession as a good practice
also in the profession is that
even for large scale AI
people select and want to have a good quality databases
to train their AI.
And actually this job of selection
this job typically for language model
the job to deduplicate the data
deduplicate the tracing data
to avoid overfitting
to avoid overtraining
of the data set, of the model
all the action that you do to select data
in order to have good performance
is actually related to data minimization
in the sense that you propose it to have
to develop large language models
so you need to have a large scale data
but you will also need to select that
and maybe if I have a catchphrase for that
is to say that data minimization
is not to have little data
but to have clean data
to be able to know
what is your data
what is your data about
are there personal data in there
if there are not personal data
you are not concerned by GDPR
but it's something that also comes of importance for IP
obviously to know where the data comes from
and I think if I take the example of the LLM
the fact that everybody is using as a basis common goal
is perhaps not completely good enough
in terms of knowing exactly what is the data
that I've been using to
to train my algorithm
and in the other range
at the opposite of the range
the work that Bloom has done
BigScience has done on the Bloom
where they're trying to really label
all the data that has been done
is something more close to what is expected
also for GDPR
and the last point that I'll try to be shorter
is how we can reuse data
and reuse data that is available online
and it's something for which it was a bit difficult
internally because
some of the people thought that
if the data has been illegally collected
we use it, it's like
I don't know in English
so like
it's forbidden
it's also a criminal offense to reuse
something that is illegal
and we came out with the idea that
it may be a big stop
to any kind of AI training
to say that
and we ask now
controllers to
to try to
determine if it is
if it is manifestly illegal
and manifestly illegal means that
it's typically data containing
sensitive data in the sense of GDPR
health data, biometric data
and you're not sure that
there have been explicit content from the data subjects
and that is
it should ring something
maybe I shouldn't use that data at all
if I have health data coming from
I don't know where
it's also a problem
if I reuse
copyright data that I know
I reuse all the Marvel movies
to train an AI
probably you should know that
it's probably not completely deleted
if you haven't talked to Marvel
and the same for for instance
if you reuse data from data breaches
data that has been leaked
probably it's also illegal data
in the first place
it's manifest
that it's illegal data
and you cannot train an AI from that
No worries, no worries
and with the GDPR being such a key piece of legislation
it's important to get these things right
and one last quick question on GDPR
before we move on
because we're running out of time already
no, no, don't worry
do you get a sort of
get out of jail card with these rules
so the idea that you know
you didn't know
you weren't aware
particularly if you're not
you mentioned the first case
of working with personal data
but not
you know
not necessarily having the aim
of processing personal data
do you get
especially as a startup
do you get
that kind of
leeway?
I don't know
technically no
we're issuing guidance
to guide
but also
in order to enforce
we try to not enforce rules
that we haven't made clear
at first
because obviously
it may be difficult
to say okay
you never told anyone
that that was forbidden
why do you strike me
so that's why we
now we're publishing guidance
I think for the next
month
and perhaps
one or two years
we'll keep doing that
and we'll enforce
and we'll then ask
you've seen our guidance
you should know
and you should implement
but one key point
is the fact that
be careful
when you
work with sensitive data
health data
minor data
biometrics
law enforcement data
this is something
where you want
extra lawyers
to look at
what you're doing
because
if you don't
you don't
look at
what you're doing
because
and if you do
a general purpose AI
try to implement
intersection
process
of your data
training data
try to also
have
I don't know
make sure
you don't have
credit card number
in your
in your training data sets
do
do a bit of work
in order to avoid
being exposed to that
because we will
probably
be enforcing
on those issues
first
the last one you mentioned
was anti-discrimination laws
and I think this kind of
brings us nicely
into the sectorial
sectorial aspect as well
so the idea that
if you are
developing an AI
that's in a particular sector
you might be
facing laws
that are specific
to this sector
that you may have not
thought about
that aren't even
technical laws sometimes
again maybe
could you run us through
what is the big
state of thing
around that
what are the
what are the key sectors
would you say
that are particularly regulated
that people
would say
well
so I guess
here again
it always depends
on what you're building
what's your product
what's your
model
of course
if you are
in the
aero industry
you're gonna
get like used to
being regulated
and you're
you have
a lot of
before even
being able
to be a partner
to
yeah
you're gonna have
a lot of regulation
on that side
but
medical products
if you're building
robots
that are here
to help
surgeons
of course
you're gonna
like we're gonna
have a particular
regulation
on this
because of course
it's like
not just
product safety
it's like
product safety
for like
touching humans
so I think
medical products
cars
are also
an industry
that is already
highly regulated
and where your
like autonomous
autonomous car
there's a lot of
discussion
on regulation
as well
and
that also goes
back to
personal data
because
a lot of
the
autonomous cars
are actually
like
collecting
a lot of
personal data
which is also
like something
that goes back
to GDPR
the finance sector
as well
is very
highly regulated
and
I think
I also wanted
to mention
that
in the digital
services act
that is actually
not
it's related
to the protection
of personal data
but it's not
in the GDPR
because it's
in the DSA
it's a
regulation
on
recommend systems
so
basically
if you're
training
a model
if you're
doing machine learning
and you're
trying to know
which are
the advertising
that are
gonna be
the most impactful
to the person
you have a provision
you have a few provisions
actually in the DSA
that are regulating
recommended systems
so
my point would be
very like
it really depends
on the sector
and the product
you're building
and actually then
to try to
ask yourself
okay what I'm
what kind of data
I have
what kind of regulation
that may already exist
and apply to my business
and then
pretty much
yeah
try to
to
to be as
you know
as careful
as possible
could you talk
a bit more
as well
on the
to again
the anti-discrimination laws
that you
or regulations
that you mentioned initially
so
when I said
that this might apply
to sector focused things
it's because
I imagine
it might be particularly relevant
in sectors
of HR
of finance
for example
yeah
what is
the state of things
here as well
and
am I right
in saying
that it's sector focused
for example
so
this whole
but
the
state
based
which
is
a
the
sort of
babies
the
workers
there
you
are
basically
we
don't
that
we
actually
you're
even
can
Perhaps it's a good example, HR, perhaps to move also to AI access.
HR software are not regulated right now, not really.
You have anti-discrimination law to your HR processes,
so you should not discriminate people when you're hiring people
or when you're evaluating someone in your company.
And there would be some AI systems,
there is already AI systems that help HR services to filter the images they receive,
for instance, or to pass tests when you have different applicants,
and you have an automated analysis of the test.
And this is where existing law exists, anti-discrimination.
If you're able to prove that you've been discriminated by a company, you can act.
But it's very difficult to prove.
If you...
You can challenge it on the GDPR way, saying,
OK, you are processing my personal data to do that, I do not agree.
So you have the question, OK, can we rely on consent when you're an applicant?
Well, obviously, you would consent to anything the company would propose you
because you want to apply to that company.
It's very rare that you say, no, I want to oppose that,
and the company would say, OK, but you don't come in.
And the AI act will provide some kind of
legal protection in some way, and tools also for the regulator,
because AI systems that are used for HR will need to comply with a few rules from the AI act,
and in that sense, it will really complement and perhaps operationalize for us,
operationalize what we expect from this software in that context,
and try and maybe make it more straightforward than deducing from GDPR all the rules that may apply
to an HR software.
That's a very smooth transition into the AI act, thank you,
which we will talk about because it is an important topic at the moment.
So speaking of the AI act, my first question would be,
who needs to worry about the AI act?
Is it all AI companies?
Is it only companies working on things that are very risky?
To what degree should you worry if you're building an AI today?
So first of all, I think,
we need to take a step back and just say that the AI act is a regulation
that is currently being negotiated at the EU level.
We are at the final phase of the negotiations.
We have, so it's called the dialogue,
so the institutions at the EU level are actually making sure that they all agree on everything.
And so it's still in progress.
There are three things where there are still,
a lot of discussions,
but what you need to know is that the AI adopts a risk-based approach.
You have risks that are unacceptable,
and there you have three main topics where you have behavioral manipulation,
you have social scoring,
and you have real-time and remote biometric identification systems.
And all three are considered
in acceptable risk,
and they're all forbidden.
And then you have the high-risk systems with eight categories.
I'm not going to go through all of them.
But basically for these risks,
we consider that you can still do them,
but if you do them,
you need to put in place compliance,
compliance provisions,
compliance strategies inside your company.
And then we have the third category,
which is the lower level of risk.
The lower level of risk,
where you pretty much can do whatever you want
without having loads of obligations.
So I think this is very important to keep in mind,
because here,
basically if you're not doing one of the three things I just mentioned,
social scoring or real-time biometric identification systems
or manipulation of people,
basically you might fall into the high-risk system.
And then you have specific obligations.
But to answer your question,
basically you really here then need to look at the provisions in the law
and see if you fall into them.
It really depends on what you're building
and what you're developing.
And then if you don't fall into these categories,
you're probably going to be in the acceptable risk section
where you can pretty much do not have obligations.
Would you agree with Antoine
just so that there's some kind of...
Yeah, exactly.
So we wait for the final list of annex 3 of the AI Act.
We list the high-risk system.
We mentioned some of those.
I don't know, AI for the justice administration,
AI for HR,
some other biometric system.
So you have a list that I think the co-legislator
wanted to make it as specific as possible
in order to make it clear for companies
about who is concerned and not.
But the list of the parliament is longer than the list of the council,
so we will wait for the final list.
Maybe what I could mention is,
for instance, one of the things in the list is
an AI system that is used for exams for students.
It can be high-risk if it's something that decides
if you have your job or not.
OK, it's high-risk.
It could be high-risk.
What I would say is,
we at the CNIL, we issued guidance in September
about the very same type of system
and how you can comply with GDPR.
So it doesn't mean that if you're not high-risk,
you do not have additional provision from the AI Act.
It doesn't mean that you can do whatever you want
regarding the AI Act.
But if you process personal data
and if it's set for it with a large effect
on the AI Act,
then some people of you need to comply,
which is what happens.
Thanks.
And this will be my last question
before we open to the room.
My question is this.
So I think the wider discussion around the AI Act
for start-ups in this space is obviously
that it's hard to prove some things
that we don't know.
It's not a common law.
It's not forward thinking.
At the start of the day,
how can you make specific books
that you can secure that you are future-proof
in terms of regulation?
Are you thinking of things
like making sure you're transparent on XYZ
or, I don't know, having a certain, you know,
having some kind of or logs
or things like that?
Are there very specific processes
that you can put in place
to make sure that you're taking those boxes
that are likely to be the ones that fall
in upcoming legislation?
I think the personal is beginning to document.
Normally, it's a good professional practice
to do that.
But really document the data.
Document the source of data,
what has been done,
and be prepared to answer those questions
and to answer the questions such as,
do you have personal data in the test?
Do you have the right data in the test?
Also, be able to have enough documentation.
Because just also when you use,
a lot of AI systems are using existing data
or API.
So if you use ChatPT API,
if you use YOLO for computer vision,
be aware of the limitation
of that kind of either generic data sets
or generic model.
And be critical if you want to use those
for sensitive use.
If you want to use YOLO
to implement that on cameras,
for law enforcement,
to arrest people,
you need to be aware
of the limitations
of being able to explain
all the bugs that you've been using.
If you don't have enough information
on YOLO to do that,
perhaps use a different data set
or a different model
or even perhaps try to collect
your own data for your model.
Maybe, here again,
try to take it back.
I'm going to say something
that is not echoing
what we heard this morning
from Eric Schmidt.
So I think,
you know,
having looked at regulation
on technology law
for quite a long time now,
I can safely say
if we regulate it in Europe,
it's much more efficient,
it's much more valuable,
and it's much more easy to apply.
Before in the 90s,
we had directives
where many states could pretty much then
apply the laws in national
with their own interpretation.
And we didn't have, like,
a strong framework for people
to actually just, you know,
apply across all EU member states
the same law.
And we should be thankful
that we have laws
at the EU member states.
Otherwise, you would have to deal
with 27 different national laws.
And I also think
it's another, like, point.
We're stronger together.
We saw that
with the GDPR.
When we adopted in Europe
the GDPR,
we had an impact globally
on that regulation.
And what was the success story
of the GDPR
is that now pretty much
all principles in the GDPR
are applied all over the world.
So basically,
the law we created in Europe
is basically the standard
that every company around the world
is pretty much applying.
So I think here
it's a good success story.
I think the AI activity
is not as overreaching
as the GDPR.
So its impact would be different.
But yeah, I'm a fan of Europe.
And I think EU regulation
is actually a strong thing
and much more effective
for businesses.
Thanks.
Thanks, both.
Are there any questions?
Yes, please.
Hi, everyone.
Hi, Daphne.
Nice to see you here.
Well, thanks a lot
for what you just said.
I think we should defend
our institutions
and avoid spitting
on these institutions
to provide the very much rule of law
that the democracy is built upon.
But probably why
some of the external,
the outside stakeholders
like startups
are afraid
of the incoming regulation
that they label as bureaucracy
is because they perceive
that the way in which
the rules are going to be enforced
and manipulating
for example,
to an other regime
could be seen
as bureaucratic intensive,
comprehensive compliance.
And my question to you,
Bertrand,
and in general,
and we have Guillaume Bavarin here,
Lachaillet coordinators,
what could we do
to advance in practice
a single point of contact
for startups
that deals with data,
that deals with algorithms,
that potentially
deals with copyright?
So, through guidance,
for example,
could you start
a working group
bringing together
who does measurement,
and others
to discuss
and to engage
in a comprehensive
and comprehensive
compliance
to ensure
that we are
able to
discuss
and to engage
with the ecotestories
gathered here
in a way that can
reassure them
in terms of
that these regulations
that are coming
will create
a level playing field
to be eventually
a sort of competitive advantage.
Is this something
that we could hope
is already there?
And what should we do
to anticipate
as smart as possible
and it's already very smart?
And I said that
because before
your position
you were
the actual
AI coordinator.
Yeah.
So,
I think
it's
there is
perhaps two
two elements.
One is
once we have
the AI act,
one of the effects
of the act
on the regulatory
point of view
will be to
how we coordinate
our efforts.
Because the use
of AI in the financial
sector will be
obviously
by the financial
regulator,
the use of AI
in the health sector
will be
obviously
by the health
regulator
that already
authorizes
medical device.
Maybe other
the HR system
will be
actually
in the
equation
of how we coordinate
the efforts
of the regulator.
Now,
we make sure
to companies
that they will not
have three different regulators
asking for
different rules,
different regulations.
I think
we all agree
that it's necessary
and
it will
to be launched
in 2024
when we know
which are
the high risk.
So,
which are
our
Dutch
equivalent
of the
risk
and
the
role of
coordination.
So,
they are
doing that
and
there will
be
the
single point
of contact
or at
least
they will
be able
to
orient
the company
to
.
What I also
see is
that
you have
startups
that want
to sell
AI
to law enforcement.
Okay,
you have some
that say,
okay,
I don't want regulation
but most of them
they know
that it's sensitive.
They know
that they need
to understand
the law
and they know
that there is
a specific
legal framework
and they invest
in that.
The same for health.
So,
we
I do not
believe
in a system
where,
I feel,
that it will be
for the best
obviously
some of these
regulations
are there
for good reasons
and
you need
to
obviously
if you want
to go to
a risky
business
that's high risk
you need
to
know
what kind
of legal
landscape
applies
and
apart
from AI
actually
that will
give you
a better
solution
for your
health.
Thank you so much
for the talk
Suzanne and
Bo-Tang.
Maybe
a question
for you
Bo-Tang
and
recognizing
my French
is limited
and I listened
to President
McClellan's
speech this morning
without English
subtitles
I heard
that he said
the French
government
is kind of
working on
AI legislation
and regulation
that we're seeing
with the
EU
Act
right now
that the
possibility
that might not
get finalized
this year
you know
there's a lot
of debate
on information
model rules
and all these
hurdles
do you think
that you know
the France
and other
countries
will kind
of start
crafting
some AI
legislation
you know
because
the Act
has a lot
of
initiatives
with OECD
about how
we can
regulate
but more
in a
co-regulation
approach
foundation
model
I think
this is
what we
want to
push
in early
in 2024
but that
doesn't
mean
maybe
probably
try to
replicate
the
Act
now
well
I think
the
only
way
we can
actually
fix
this
is
to
give
you
the
opportunity
to
try
to
do
some
research
and
to
integrate
with
the
EU
Act
work with companies about documenting that, the live data to meet that.
So you may have a few laws perhaps on health, perhaps on finance, that will be inspired by AI Act, it's probable.
A few days ago there was this news, it might be true or not, the French police uses a software that allows to do facial recognition.
We will see if they use it or not. How do you see regulation in government states?
Because you talked a lot about companies, private companies,
how do you see this kind of regulation happening within government and how do you control that?
So it may be different in different countries, including Europe, because there are directives and regulations about law enforcement.
In France, they need to control the CNIL, and in many cases they need a law.
The example of the Olympic Games is that they need a law,
and what has been reported,
especially in the press, is typically something that is illegal because there is no law allowing them to do that.
So the main checks and balances will be the parliament and CNIL in some aspects,
because we have investigation towards what the government is doing.
Maybe a follow-up question on the first question for Susan on the idea that in the AI Act there is an AI office,
and there has been negotiation around the role of the AI office,
and whether a lieutenant is going to settle two weeks ago.
So there is also the possibility to have enforcement and investigation at the AI level.
What is your take on this aspect of the law?
So enforcement is always a tricky question, especially when we're dealing with actually 27 member states,
and we're actually discussing about this.
I think one of the key issues is having one law is a very good thing,
having 27 people making sure that that law is well-applied is another thing.
And I think national authorities are very much important,
because you cannot, as a startup, go to DigiConnect and say,
oh, I'm sorry, I didn't quite understand this piece of article.
Can you please help me?
So pretty much what the national authorities are doing already,
trying to help companies and startups better understand potential applications for their businesses is very crucial,
and we need more money on this to avoid having enforcement problems then.
And then we have the enforcement issues when we're dealing with applications of the law
at the national level, with the EU level,
and I think we saw a lot of problems with GDPR enforcement,
with the Irish Data Protection Commission being the bottleneck of Europe's enforcement.
So I think the new upcoming regulations, including the TSA, the AI Act,
are trying to overcome these issues.
And I think a good national, or a few good national authorities,
is working and helping compliance is very crucial.
And then having a coordinated enforcement system at the EU level is also very important,
but also still having some national powers to national authorities is key balance, I would say.
Do we have time for a last question?
No? Okay. Never mind.
Thanks, everyone.
Please join me in a round of applause for our fantastic panelists.
Thank you.
Thank you all for this great panel.
We'll be back at 2.50 sharp for the next session.
Thank you so very much for your time and your insights today.