good afternoon everyone welcome to our
spring quarter
ikea seminar series um before we
i introduce our speaker today i want to
go over
a few things about this seminar um
for you know for all the audience
i think uh this seminar series is also a
class for
some of our students so for this of
those of you who take this as a
class i would like to remind you that
you need to complete
uh seven assignments through canvas
out of the eight seminars
so so you need to submit your
assignment and answer i mean
within canvas for questions
uh we would like to you to hold your
questions till the end
later and until the q a session
if you have a clarification question
you can read to your hand otherwise
we'll leave the questions to
the q a because
we have also an audience from outside
and for you um that
you you can only raise your question
through
the q a function but for our students
uh we would like you to have your
questions
through chat the zoom chat function
and you can post those kind of questions
at any time and not necessarily by the
end
so that's a few things
to tell our audience and now
i would like to introduce our speaker
so today we are very pleased to have
professor
sharon d as our first speaker for for
the
spring seminar series uh sharon is
right now an associate professor in the
department of civil
engineering and engineering mechanics at
columbia university
um and in new york city
she received her phd degree from the
department of civil
environmental and geoengineering at the
university of minnesota
and did her post-doc door research at
the university of michigan's
transportation research institute so
sharon's research lies in at the
intersection of game theory
dynamic control and machine learning
with applications to emerging
transportation system operation and
control
shared mobility modeling and data driven
turbine mobility analysis
she directs the dtac lab and serves on a
committee for the
smart city center in the data science
institute
at columbia university she is a recent
recipient
of the prestigious national science
foundation career award
and a winner of the transportation data
analytics contest
from the transportation research board
her talk today
is on dynamic driving and routing games
for autonomous vehicles sharing
the zoom room is is yours
thank you so first of all and
i'd like to thank professor john for
hosting me today
and my name is sharon d and i will talk
about some ongoing work
in my group using the mean game and this
is a joint work with
my students hu chen and my postdoc
ron yeshu and also students from uh
from mass guang huang and also
professors can do and also he turned
from maths and computer science
so professor john just talked about my
career path
so now i just quickly go through it so i
did my postdoc
for two years at the university of
michigan before i moved to colombia
and then at the columbia i direct this
so-called
detector lab so the full name is data
and innovative technology driven
transportation lab
so i call my students in my lab
detectors
and this technology i want to focus on
today
is the level 5 automated vehicles or we
call it autonomous vehicles or
driverless car so there are multiple
names attached to it
for technology it's actually a
double-edged sword
which means it has the positive and
negative sides
from the positive side the author's
vehicles are claimed
to improve the traffic efficiency and
safety
and on the other hand when they interact
with the human drivers
on roads so we call the mix traffic
then and it may produce these
unprecedented consequences
like what do we observe that the google
self driving car have crashed into
buses and also other cars and also three
years ago the uber car
killed a pedestrian crossing the street
at night so many researchers focused on
transforming the technology for social
good
so because i focus on this autonomous
vehicle
and let me show you a video to
to see how this technology may
need to be improved uh to better
interact with human drivers
and the traditional approach to model
self-driving cars is to trade them as a
control
agent so what i mean by control is that
they
they follow some mathematical
formula basic equations and then they
they are dictated by these dynamic
systems
such as the ordinal differential
equation however
in this video i will show you when the
way more current put onto the public
roads
and then you can observe some very
awkward phenomena
so this video was uploaded by some
twitter users
so you can see this card this
webcam is recording is how we move car
try to merge through the highway so you
can see it's
left the blinkers on
and i i think this is in california
and all the human driven cars in the
neighborhood were able to
merge in but then this way more curve
failed to find a gap and then eventually
he had to actually get off the
run and gave up using the highway
so from this maybe we there are many
these kind of
videos and actually in youtube you can
search
from the human perspective probably we
don't really understand why they they
behave
either conservatively or or differently
from what they expect
so that's why we project that in the
filter we should model autonomous
vehicles as
ai agents so ai is artificial
intelligence
so what do we mean by ai so first of all
they should
be a rational utility optimizing agent
and so they should behave rational as
whether humans behave
in most situations and then they may
play a
cooperative or non-cooperative game
with the road users in in their
neighborhood
and they should have developed this
learning and adaptation
capability to navigate in a complex
traffic environment
and so that's why i believe in the
future
the uh av which refers to the autonomous
vehicle and age will refer to human
driven vehicles
and they will play a game on these
public roads
and they really need a game theory to
empower the driving intelligence
of the autonomous vehicles
and in this talk i aim to to answer
several research questions and basically
if we in the field
we bring multiple autonomous vehicles oh
a large number of
autonomous vehicles to the traffic
networks
the first question we want to understand
is
what are the optimal control problem we
can formulate
for uh for an autonomous vehicle or av
and here i will i will talk about how
how will your scheme series model
each car speed choice and also the raw
choice
however we know that in a row network
there are many other cards that also
exist
so what if everyone tried to optimize
its own
while others still doing so
simultaneously then this
forms a game so we can not just the
model one av in an isolated fashion we
have to look them
look at them in a multi system a
multi-agent system
and also the third question i aim to
answer is if it's a mixed traffic
scenario
so how the human drivers should be
modeled and how they
interact with the autonomous vehicles
to answer these questions so in this
talk
um i will uh introduce how we borrow the
game theory
control and also the machine learning
um to to study these three research
questions
and in the rest of the talk i will
touch on each component one by one
so let me describe the set of uh setup
of
our work and assume this is a rain road
and then there are several avs that are
running on this road
and through basically in the first
component i will introduce how we use
the game series to control many cars
in an efficient and scalable way
and then this game theory model turned
out to be
kind of like a neo-traffic flow theory
to um
model how avs will be controlled
on roads in the future um
first away because we assume that aih
gen
which means they should have objective
function or we will functionally need to
optimize
and then we introduce this objective
function called j
so it's composed of this f function
over time so we call the cost function
so this cost function depends on the
speed and also the position of the
control the car
and it can contain
the terms that represents one car's
running cost and congestion and also
safety costs
even we can include terms that reflect
the emission
and this v of t represents the terminal
cost so what's the cost that we want
this car to achieve at
the the end of the planning horizon
and for this uncar differential game so
each car solves the optimal control
problem
and and at the equilibrium so when other
cars
hold their driving strategies fixed
then this car strategy which we call it
what
they end with respect to time
should achieve the minimum travel cost
so that's the meaning of the equilibrium
in this game however
when we increase the number of aways on
the road
when we when their penetration rate
increased in the future
so this problem actually becomes not
scalable so how we can control so many
cars
and move them on roads simultaneously so
that's
something like impossible work
so what we did is instead we borrowed
this concept
of so-called mean field game so mean
field game
actually aims to model complex
multi-agent dynamic system
when the number of agents involved in
the system goes to infinity
so in other words so this game actually
is a non-cooperative nice game
with the player number go to infinite
and and this game was proposed by a
group of
economists and mathematicians and then
and since then they were able to explain
many um
phenomena for example this mexican wave
so what mexican will describe is one
in a big stadium a lot of
audience there when each of the
individual
agents they do this stand up and stay
down movement
but aggregately you found out that their
collective behavior generate a
wave that moves horizontally across the
state
stadium and then this game also
can successfully explain some natural
behavior in animals
groups such as fish schooling
the reason that this game becomes so
successful and
widely adopted since its inception is
because
it can link this microscopic agent-based
behavior
to the macroscopic collective behavior
of a
mess okay so our question then how do we
implement this idea to the traffic
control
so this is what we want to do using
so-called mean field approximation
so still let's come back to that uh
raindrop scenario
um the for the eagle current we want to
control
its speed and then we assign some
predefined
objective function j but when we apply
this mid field
which means when we increase the number
of agents on this road
so then actually this car
because every car becomes like a
homogeneous agent
and then they can ignore their index and
then they trade this car we try to
control the heat
as a generic car the cost function is
a same however
instead of interacting with every
individual car so this car can only
perceive the traffic density in its
neighborhood
and then this game actually is
transformed
and we just need to calculate the
so-called new field equilibrium
which is characterized by yo star and
rho star which is the velocity
and the traffic density in time and
space
so we call the density field and the
velocity field
and to solve this problem it has to
employ so-called forward-backward
structure
we start from aging the dynamics so
individual car
they they try to react to the older cars
in the neighborhood and
minimize objective from function j
but once we apply this mean field
approximation
then their objective actually only
depends on its own
speed and also the traffic density then
assume
we know the speed of that current we can
feed it back to so-called mass
dynamics which is the uh
key plus rho x equal to zero so this is
the flow division equation we always use
uh in the traffic flow theory
number when we gave you then we can
solve the row
and then we fit this row back to the
agent dynamic
so we updated the traffic density and
in the traffic environment and then the
the car will adjust their speed
accordingly and then we do this back and
forth
eventually reaching equilibrium and then
basically we need to solve two
equations once the hjb equation saw
backwards
that actually describe how individuals
saw an optimal control problem and then
on the left hand side we saw
so-called fpk equation which is of the
forward
to derive this traffic density
so when this process goes back and forth
until it
converges in the rich so-called midfield
equilibrium
so you escape all the details but let me
present the results
we compare the scenario when the
avs are controlled by this midfield game
versus
when they are driven by the lwr so lwr
is a first or older traffic flow model
the weather used in transportation so it
basically
it you assume a fundamental diagram and
then you
assume the density and velocity field
evolves according to to flow
conservation equation
and as we mentioned we need to first
define or predict finance cost function
so we found out that the lwr actually
can be reinterpreted as a mean field
game with this constant function
so your rho minus u divided by um
squared so um is maximum speed so this
difference actually means that um
under the ilwr model cars try to their
goal
is to drive their speed towards the
electron speed which is defined by some
some
fundamental so based on the
understanding of this
lwr objective function so we either one
term
rho j minus rho divided by rho j rho j
is the jam density
and so this term actually represents
that the trap
the car also try to avoid the crowds
they they try to ensure that the
extensive deviate from the
gem density as much as they can so by
adding this
additional term we found out that the
the the traffic flow
uh exhibits very different patterns so
that's focused on the figures
on on the bottom so the left
one is
the how the row the traffic density and
the speed evolves so this under the
midfield gain
the right hand side is when the density
and the velocity evolves
under the lwr model so
you can see at the beginning the
position of the cars
on the ring road are exactly the same
in these two model scenarios so there's
way way delivery put
cars more cars in this neighborhood to
generate some
uneven distribution of the cars
let me play this video so you can see as
the time goes on
and the left-hand side of the
this jam and jam
area gradually actually smoothed out but
on the right hand side with this lwr
model
a shockwave emerges here
so from the upper animation you can see
that
so this area still we can see the
traffic jam
sorry now this area this area so the
color of the
uh this rainbow represents the density
the
darker means the the more more congested
but on the right hand side for the
midfield game you can see the
the the traffic eventually move towards
the
uniform flow so that tells us that
midfield games obey carefully designing
the
reward or objective function of the game
we can actually better manage the
traffic flow
in the future when we have the
autonomous vehicles
and also we found out that lwr can be
treated as a special
class or mutual gain and and with the
objective function
um being the everybody trying to move
towards the equilibrium speed that
is defined by fundamental diagram the
second
actually lwr can also be treated as a
myopic neutral game which means
when if the car
only optimizes velocity based on the
density in its neighborhood
at current time so they only get lwr but
for midfield game
so cars can basically anticipate the
future traffic condition
and adjust their speed accordingly
so this is the first part of
my talk so we basically designed this
midfield game to model
how autonomous vehicles can
can be controlled in the future and for
the
second part of my talk so i will
um move forward that
from this rain road to the
transportation network
and i also similarly i will compare how
mean field the game
and will how the cars driven by midfield
game exhibit the different
patterns from those driven by lwr
i will escape the details but let me
introduce the setup of this
model so this is the role network
we try to look at so it's a very simple
small network we call the bridge network
so it
contains four nodes and weighs the five
links
and we assume there are traffic demands
coming
from the road um one and two
and the objective function we use here
uh contains two parts the running cost
and the qr cost
so the running cost composed of three
terms
and the first term actually defined kind
of
kinetic energy the second is the safety
so basically the car try to avoid the
congestion and constant part when you
put it inside the integral from zero to
t actually it represents the travel time
or driving efficiency
and we also introduce a killing cost so
the the reason
we have we're curing causes because when
the flow
from node one the two may meet at
another's weight
so then if the the incoming flow exceeds
the capacity of the noise rate then
some q may form here and i will compare
how the traffic pattern evolve under lwr
and the midfield game models
and here i will show you two figures the
left is the density
row the left the red is the velocity
yield as time moves on
and i mainly showed how the flow along
this path
134 evolves as time progresses
so yeah we inject the traffic demand
around this time and if we
run it under the midfield game so you
can see the pattern
is quite different so i want to point
out
the difference which is here the the
peak
um that emerges under the lwr model
so this is because when the flow coming
from the node two
meets the flow from node one at the node
three
and they then they they're
some of the flows exceed the capacity of
those rates so that's why there's a cure
um basically sitting there cars
by the way assume this first thing first
account and
because of the kill so the velocity
actually dropped
in the similar area for for the
for the velocity but if you look at the
mean field game
this doesn't exist so the reason is
the midfield game actually can um
control
through a midfield game we can control
how fast
a car drives on a road so
we deliberately actually let the car
coming from node
two to three slow down so that they
don't have to meet the
the node the flow from node one
simultaneously
so that actually made it get the slow
peak
and then the whole traffic pattern is
mitigated
so from this we can see that in the
traditional traffic
model like lwr it assumes the car
speed is controlled by density but now
we introduce this game that
will enable the car to further control
their driving
velocity on each road so then that
provides a lot more freedom
to to the aby control and also can help
me to get the traffic food
okay so this is the second part of my
talk
and first i i introduced what's the new
equilibrium
it can achieve in the future if we
replace all the cars on the road
by autonomous vehicles and how they can
move on road networks by both
selecting their spirituals and road
choice
so the in the last part of my game uh
the talk
i will
introduce how we can leverage the data
that collected nowadays from the
avs and also existing sensors
to better model human driver behavior
and how humans
interact with the autonomous vehicles so
these are ongoing work so it may be a
little disconnected
from the first two components
and first i will talk about the
sorry
we leverage there's way more car
data that was publicized i think two
years ago
so basically and the waymo um
shared some of the data collected from
their camera
and also the light that mounted on the
top of the remote cars
and the left is the camera image
and then the right is the light up on
the cloud
and then we aim to leverage this new
data format
for driving behavior model because the
the purpose of for
wemo to publish this data is to
actually for the communication community
to better develop
efficient algorithms for the
object detection and tracking but uh um
we think that maybe we can use this data
to also
uh better understand how we mokkai is
programmed to drive
and so for this work
i don't know why this box shoulder again
yeah so a way fade in
the basically the right hand side of
this video was captured from the camera
of the
remote car so from this camera we can
extract
the distance from the
leading car which is a wider car in
front and also speed difference
and also we use these images and input
and faded to a machine learning tool
toolbox
here we use long short term memory
model and then we try to predict the
acceleration rate that the memory card
will
adopt
so you and here is actual position of
the wemo car
the orange one is the
predicted one the red car is actually
the
wider car in front of it so we use this
can
capture actually the car falling
behavior of this remote car
with this reasonable error allowance
and through this i want to show that
because the image data and also the
lidar points data are not traditionally
handled
by the transportation community when we
try to model cars
car falling or land changing behavior so
that's why we have to resort to
the machine learning models and however
another message i want to convey is that
so this machine learning or black box
tools sometimes may fail and
uh and also it
seems like a conflict from the domain
knowledge we have developed
for example in our transportation
community and
there are like a large number of models
that have been developed
developed in the past decades and so how
we can
reconcile the machine learning model and
domain knowledge
i think it's also a promising direction
we should explore machine learning can
help discover some mechanism
from the data that collected from the
domain
and however they request a huge number
of representative samples to trim
on the other hand if we have already
discovered certain
certain patterns which satisfy some
physical constraints
then probably we can also supply those
domain knowledge into
machine learning tools to better train
this model
so let me compare this so-called we call
the physics-based the model and also the
data-driven model a little bit in this
diagram
so the x-axis represents how much data
the model type uses and the y-axis
represents the how much
scientific theory the uses you can see
for the existing physics based model so
all the model that developed
mathematical model development in
transpartition we call
it the physics based model so they use
very
small very small number of data but they
highly
rely on our scientific hypotheses
on the other hand the machine learning
models they highly rely on the
all purely rely on the data and without
even
knowing what's the physics hidden
in the data set and and now there's a
like a growing area called physics
information learning basically try to
leverage the advantages of both physics
mode of the machine learning model
and so that we can trim those machine
learning models
using those knowledge of the physics
and let me give you an example how this
idea can be applied
to traffic flow traffic state estimation
and actually here
[Music]
i don't know
sorry let me reshare my slide it seems
the
sharing yeah maybe we can pause and then
you can
re-share so we can move that yellow box
uh
i was doing uh myself and christina
we're looking at this and it looks like
it's a setting in zoom
um that needs to be changed
and it's actually not an easy setting to
get to it has to be done with the zoom
settings themselves um it looks like
this may be something that
just is a new thing that we're coming
across
uh possibly due to the update
but um christina can you uh
privately share that link um
to sharon and we'll pause the recording
right now privatization
so she can go ahead and uncheck that
setting and zoom
yes we're sorry for the interruption
and so let's come back this is here i
want to show you how we can
trim a neural network to approximate the
solution of a partial differential
equation
and this is the uh partial differential
equation we try to focus on the rho t
plus nx
of rho equal to zero so this
an x is a nonlinear differential
operator it can be any form that depends
on
row and then rho can be a
any physical value we are interested in
in the traffic flow usually by entering
density or velocity field
defined about x and t the time and space
domain and then we define a residual
which is just the left-hand side of this
equation
and if our goal is to train a neural
network
that approximate the solution of this
rho of tx then they either the
this optimal solution the residual
should be equal to zero
so how do we trim a neural network uh
because we know that
the key of training on your network is
the loss function so here we define loss
function
as two components the first
is the mean square error of the observed
data from the predicted value
and the second we introduce a new term
which is the
deviation of a predictive value on the
collocation points
so the collocation points actually are
related to the physics
and i'm trying to refer to and i will
explain to you in the next slide
here i just use a simple example so the
um the pde model we try to solve is a
lwr listed on the left
top top left corner so rho t
plus the rho yo with respect to x
equal to something so usually your zero
on the right hand side but here i
introduce
a viscosity term to make the solution
smoothed out and we use the windshield
fundamental diagram which is uh
basically they found a linear
monotonically in
decreasing relationship between the
speed of yo and the density row and we
define some initial
and boundary condition we assume this is
our ring load
so you can see this heat map actually is
the
solution of the rho tx the density
field in x and t
space time time space there well and
here we we define our ob observe the
data
assume we place some loop detectors
at a certain locations so because the
loop detects
are fixed so that they can collect the
data continuously
along that direction and then assume we
can
observe all the density data
in those specific location and across
the time
the collocation point here is actually
you can randomly sample
those t and x from this field
defined by the uh the x from zero to one
and the g from zero to three
and uh on the collocation points we will
calculate
the the value from the pde model assume
we know
this is the physics we want to use to
trim our neural network
and then we define a loss function that
is composed of these two components
and the first mse row is the deviation
of the predicted value from neural
network and
the the deviation from x the observed
density
uh that collected from loop detectors
and msef
is the the residual or the difference of
the
physics value we calculated on these
black cross points
and then then we train the neural
network and we produce the whole
density field you can see the the lower
bottom this figure can
actually that this heat map that
represents the predicted density field
is
almost the same as the exact
raw tx that solved from this lwr model
so this is a demonstration assume that
we know the
uh exactly the physics that generate
data
then we can trim a neural network that
produce the
uh the solution of the pde exactly
using those only some sparse
observe the data we also apply this work
to the real world data set which is the
engine same data and
the engine same data was collected on
two highways in california
and basically they mounted some cameras
on
the near some buildings by the side of
the highways
and then extract all the individual
vehicle trajectory
in a specific segment for for a specific
time period
and and then we using all the
vehicle trajectory data we can
aggregate them and calculate the density
field here
represented in this heat map and we
we call it rho tx which is the density
field along
the time basically to 45 minutes and
access
roughly 600 meter segment
and assume we place several sparse
loop detectors along this segment and
here i will mainly point out the one
funding we have so on the bottom of
figure
the x axis is the number of loop
detectors so we
increase from the 2 all the way to 18
basically we even distribute all these
loop detectors
and we train a neural network and we
we use also some lwr model as the
physics
to help out us better training our
network and then we do the prediction
and we found out that as the number of
loop detector increases basically the
more observed data we get
actually the less we rely on the physics
so the beta is the weights uh in front
of the
the loss loss term uh in front of the
physics and alpha is a loss
loss term the weights in front of the
the data term
so when this the ratio of beta over
alpha decreases which means
when we have more and more observed data
we more rely on the neural network
to predict the traffic density
and less rely on physics however when
when
we only have two or four uh
loop detectors then in this case it's
very hard to trim a very good neural
network because usually it's
basically requires a huge number of
representative data
then in this case physics actually play
a better role if
we have a more accurate physics we can
still use
so sparse data to make a better
prediction
so that's all about my talks today let
me make a summary
and so my research mainly applies the
game theory control and motion learning
to better understand the traffic
patterns
uh in with the emerging transportation
technology
but and there should be some convergence
about these different fields
the one question very interesting answer
is what are the new equilibrium in the
future
when we introduce this uh autonomously
because of the
the other role user that we've never
seen or modeled before
and the second question i'm interested
in answering is
how we can leverage the technology
to better manage our traffic flow um
so these three fields should not be
isolated
and i and hopefully we can also merge to
the game serum machine learning say
to close the loop as i think now in the
game theory field
people are trying to also study this
learning in game it's not
a new field but i think more and more
people are trying to look at the
convergence
between reinforcement learning and and
also
the multi-agent reinforcement learning
and how agents play games
in the system and on the other hand
whether
the machine learning can be also
incorporated into control field
so-called learning-based control so
assume we don't know exactly how the
the traffic the dynamics the
environmental dynamics
evolves whether we can borrow machine
learning to to learn these
evolving states of the environment
and better come up with some better
reliable control
so that's all about my talk today thank
you very much
thank you sharon um i think uh we'll
open up to uh
now is the asex session that will open
up two questions
again i would like to remind you know
for our student audience you can use
raise your hand
to ask your questions and for our
outside audience
um you can use your q a feature
to ask your questions so as a host let
me
take advantage of being a host and ask
the first question so sharon you're
being you know very successful
navigating the early career of a faculty
we many of our students uh aspire to
become
academics and faculty in the future
maybe you can
use the you know the first question that
perhaps you can share your experience
as assistant and then and now associate
professor
and how to uh you know to be successful
uh finding uh the academic
job and also in the first few years how
to uh
like and how to make it work
yeah thank you for this question i think
professor john you are a perfect person
to ask this because
i think you are you watched me growing
up
your business almost of my entire career
passed so um i actually i
i met a professor john when i was an
undergrad at punji university
so that time actually i didn't know what
to do i was
very uncertain about my future i but so
far i still remember
once one thing that professor john told
me
that time because i was wondering
whether i want to after graduation
whether i want to stay
in china to go to work or i want to
study abroad and and
i during the conversation with professor
john so i raised my concern that i feel
i'm too old that time like i'm not sure
whether
it's it's already is still the right
time to
to like prepare those gre and exam
and go study abroad and
even i i go abroad i may not know what
was my future like
but he told me that it's never
late and you are still young so you have
time
to to prepare and fight for your life
so i think that really
made me determined to work hard and
prepare
for everything and then even
after i came to the u.s i think
basically during my graduate school uh
specializing after 2008 because the
economic crisis and transportation
actually was
actually the graduates from
transportation that time i think had
some hard time in finding jobs i see not
only
this field but also all the fields face
that
challenge so that time i was also not
uncertain that i want to pursue
this field but i'm happy that i insisted
because um when i was in the job market
uh probably after i graduated in 2014 to
2016 near two years
actually we see a tremendous increase in
interest in transportation specializing
thanks to the advance of the
communication and also sensing
computational technology
so that's why colombia hired me i
because i was the first
one in transportation at columbia uh
previously they may
probably not think that transportation
is a
like a essential piece of smart cities
but then they see that the trend
and then they decide to influence people
they realize smart transportation
is a and like incredibly integral part
of the smart cities
so i think that my advice for you is to
never give up
and when you feel desperate then
talk to those senior and wise people for
their
advice i think sometimes they're one of
their just a sentence can change your
life
well thank you for sharing the personal
experience i think
you know 2008 it was a hard time and now
we have the pandemic so this isn't
particularly relevant
because some of our students going to
the market will find it
very challenging and as you said the
perseverance and patience
eventually will pay off then yeah so
um any uh so i've let me open the chat
room
um a couple of thanks and
no questions here so any uh questions
you can
you can okay so joke and we can
turn on your mic and ask the question
yeah it's
it's okay thank you and thank you so
much sharon for the presentation i just
had a question
uh when you were talking about the
equilibrium and you were talking about
the wave
example um how do you ensure
that the if there are parameters of the
equilibrium that they're being met
and that the uh micro scale or the agent
doesn't negatively impact the macro
so going back to the the stadium wave
that people do how do you ensure
all the wave is going around that
there's no
disruption then creating another wave
simultaneously
i'm just wondering what your thought is
on that
um you know applying this to real life
scenarios
there's gonna be a lot of disruptions so
i was just wondering what your
what your thoughts um let me
maybe share my screen i'm not sure
whether i
i get your question go back to the
slides
so for game theory i think there are two
steps is the first is to
model what the natural phenomena is
there
so then that require us to actually
calibrate
the the game so that it can the
outcome of our multi-cam matches the can
match the
reality and so that require the
calibration of the reward function or
the objective function i would
i mentioned here when we
say if we want to use this midfield game
objective function to model human
driving behavior
so that we have to to really calibrate
what's
what terms should be included in there
so what the human driver really care for
and then what's the weights among all
these terms
so that's one aspect the second aspect
is
the guy use meaningful game to control
events
and since it's under our control
basically we can
design them so that they can behave in a
way that
achieve our goal so to do that we
basically we need to carefully find a
objective function that can mitigate
the traffic flow pattern so that's our
goal
in this case basically we design this
objective function
be like here
so we add these two terms and we found
out that this
design can actually help the the
mitigate the emergence of the shock wave
and there are some work i didn't present
is
we also yo this mean field game to model
the
mixed traffic so basically couple min
field game and also
some existing traffic model actually the
arc model
which is a non-equilibrium uh
model so when we cover them we also try
to understand
how we can design this real world and
also the coefficients
in front of this reward to make the um
the traffic the uh stop and the goal
will disappear
as fast as possible so does it make
answer your question
um yeah i bet i i understand the
mitigating effect and
and creating this equilibrium basically
the question is
what if what if one of these
uh micro pieces of the macro scale
wouldn't function uh in a way that we
would like to
um you know basically not following the
the kind of vacuum situation
would this vehicle then have to be taken
out of the equation so that it doesn't
impact the wave
basically that's that's that's my
question
let's see yeah so here we make a
simplified assumption that all the
agents are homogeneous
they are rational and they they try to
optimize their
objective function and however if one
behave very different from others so
then that
we can treat it as a disturbance or
perturbation to the system
so then this agent is not a probably
anonymous animal
it may become a major player of this
system and then
completely change the the behavior of
the system
so in this case they may have to take
out that
that agent and model its behavior and
also its interaction with all other
agents and see what may happen
yeah that's exactly what else thank you
a good question
okay precious please raise your hand
okay i um so
questions um did i pronounce your name
correctly and you can
open your life and answer your question
you ask your question
uh can you hear me yep okay
so uh thank you for the talk it seems to
me like
intuitively mean field game theory might
be most justifiable
when the interactions are local and
something like uh you know
two at a time and each agent is acting
individually
in a rational way but if you have
autonomous cars
who can communicate with each other at
longer distances
or sort of coordinate with some sort of
centralized authority
uh be it some uh operator in the cloud
or whatever
uh how applicable is
the mean field game theoretic analysis
in that case
to uh determining travel behaviors that
might say
try to mitigate a traffic jam or
something like that
yeah yeah it's a very good question so
this actually comes to how this
how we um assume
the a waste behavior and also uh
when the connectivity is introduced we
call connectivity because
how we want to control them and there
are two
uh i think the debates
so for example like in the academia or
for for the transportation like a d.o.t
they
they their pictures that every in the
future
car should communicate first and then we
can have a centralized controller
to to control their movement but we can
see
from the industry perspective like a way
more uber
when they develop their cars um they are
they treat their car as a independent
agent
moving they rely on their sensors on the
top of their car
to pursue the environment and make the
decisions
and the say the remote cards
in the neighborhood may may be able to
cooperate through some cloud
communication
but it's not a centralized system for
sure
the reason is because of scalability so
if we introduce thousands of the car
imagine if we replace
every car on the road now by autonomous
vehicles so it's impossible for
us to come up with one central control
controller to control everyone's
movement
so that's why people develop distributed
control
but from my perspective is um we've got
to
wait we need to find a correct the
motivation
uh for for why the avs want to cooperate
with
its neighboring cars usually it's
because they share some common reward
function
then basically if they don't collaborate
they get punished
but um my argument is they may also
be non-cooperative so they could they
play this so-called nash game
and because if i i'm a
autonomous car and i pick up a passenger
my goal is to transport a passenger
to its desk his or her destination as
fast as possible
why would i care about reaching
social system optimal to maybe
sacrificing my travel time
to benefit others so there should be
some motivation
for for all the cars to cooperate
eventually in the future
but i feel at this moment i prefer to
treat each car as a non-cooperative
agent
that also solve their own distributed
country
does it make sense
yeah i think i think the uh only
the only thing i'd add to that is that
the cooper the communication with the
centralized authority that i was
thinking about was more
limited to just like getting
uh getting data for instance from
google maps or having
one uh car on one side of the
ring road communicate with another car
on the other side
via a cloud provider uh but not
necessarily having centralized
control per se
yeah uh i think for for example when
nowadays
everyone use the google maps for
navigation but
google provides us the background
traffic
like the the travel average travel speed
on each road
and the predicted travel time but based
on this information
as an individual driver i tried to stand
up my shot in the past
so that's why i still behave even with
the communication i still behave
selfishly
so and a non-cooperative so that's the
fundamental
assumption for us to build this new
field game
all right thank you hey um trisha has a
question
she asks does the mean field approach
require centralized control
yeah uh following up the with my
previous answer
no it's actually distributed
decentralized
control so um however we can
probably how to make an implicit
assumptions
the cars can still perceive the traffic
condition like the traffic density
through some central uh traffic systems
so for example i use google map or i
listen to radio and i know roughly
what's the traffic condition
in my neighborhood and what's the then i
will anticipate the future traffic
condition
based on my anticipation and the
information i have
so that i can adjust my optimal speed
any other questions
um if not i have a question for sharing
it
you know for the human drivers we all
have our own personalities
and you can tell that from that uh wimbo
car try to merge on the freeway
probably it's not aggressive enough to
uh
accept the gaps and therefore you have
to exit the freeway
um in the game theory modeling
if you want to have personalities for
autonomous cars
what way can you add this feature to the
model
yeah this is a very good question
so um i think this is
also like a active
field now i assume in computer science
people are trying to to
model this um hydrogenated across the
uh the autonomous driving policy
learning
now i think he might if if we
only limit our discussion to the
midfield game so probably
in the objective function when we design
these different terms like
safety emission uh congestion so we can
put different weights
um but by the many i think when i
talked with some economists they they
said that okay maybe the
the cars in the future their aggressive
should be
can be a option for for the occupants to
select
whether i want to discard one to behave
aggressive or conservative and also
some other researchers even think about
maybe the car
itself can learn from their occupants
some historical data and
automatically set up this aggressiveness
while driving so then that that i think
will make the mathematical modeling much
more challenging
and also it opens up a lot of other
potential
problems you know if the user set
the parameters to be in a very
aggressive range
and what happens if it has a crash when
tries to
emerge um would that user be more
responsible or the manufacturer of the
car or the
common driving system and so i think
it's
yeah it opens up a lot of other issues
other than just the model you know how
to model this traffic
and how to do it yeah exactly liability
ethics
unions issues
which is a good thing that means there's
a lot more research to be done
uh good for graduate students and
researchers and to take it out
yeah and also along these lines highly
discipli cross-disciplinary
so then probably we have to work with
social scientists the lawyers
and computer scientists to answer all
these
new questions
um so in closing any other
urgent questions
so if not we want to thank our speaker
again and thank you sharon for wonderful
talk and uh um you know
if there's no questions and we'll stop
here and
and again i would like to remind our
students do not forget to turn in your
your assignment through canvas
okay thank you professor john thank you
everyone
take care