MSc(Engg.) Thesis
Title: Algorithms for Product Pricing and Energy Allocation in Energy Harvesting Sensor Networks [Thesis Defense Slides]
Abstract: In this thesis, we consider stochastic systems which arise in diﬀerent realworld applica
tion contexts. The ﬁrst problem we consider is based on product adoption and pricing. A
monopolist selling a product has to appropriately price the product over time in order to
maximize the aggregated proﬁt. The demand for a product is uncertain and is inﬂuenced
by a number of factors, some of which are price, advertising, and product technology.
We study the inﬂuence of price on the demand of a product and also how demand aﬀects
future prices. Our approach involves mathematically modelling the variation in demand
as a function of price and current sales. We present a simulationbased algorithm for
computing the optimal price path of a product for a given period of time. The algorithm
we propose uses a smoothedfunctional based performance gradient descent method to
ﬁnd a price sequence which maximizes the total proﬁt over a planning horizon.
The second system we consider is in the domain of sensor networks. A sensor network
is a collection of autonomous nodes, each of which senses the environment. Sensor nodes
use energy for sensing and communication related tasks. We consider the problem of
ﬁnding optimal energy sharing policies that maximize the network performance of a
system comprising of multiple sensor nodes and a single energy harvesting (EH) source.
Nodes periodically sense a random ﬁeld and generate data, which is stored in their
respective data queues. The EH source harnesses energy from ambient energy sources
and the generated energy is stored in a buﬀer. The nodes require energy for transmission
of data and and they receive the energy for this purpose from the EH source. There is
a need for eﬃciently sharing the stored energy in the EH source among the nodes in the
system, in order to minimize average delay of data transmission over the long run. We
formulate this problem in the framework of average cost inﬁnitehorizon Markov Decision
Processes and provide algorithms for the same.
Technical Reports
Title: Energy Sharing in Multiple Nodes with Finite Buffers [pdf]
Abstract: We consider the problem of ﬁnding optimal energy sharing policies that maximize the
network performance of a system comprising of multiple sensor nodes and a single energy
harvesting (EH) source. Sensor nodes periodically sense the random ﬁeld and generate data,
which is stored in the corresponding data queues. The EH source harnesses energy from
ambient energy sources and the generated energy is stored in an energy buffer. Sensor nodes
receive energy for data transmission from the EH source. The EH source has to efﬁciently
share the stored energy among the nodes in order to minimize the longrun average delay
in data transmission. We formulate the problem of energy sharing between the nodes in the
framework of average cost inﬁnitehorizon Markov decision processes (MDPs). We develop
efﬁcient energy sharing algorithms, namely Qlearning algorithm with exploration mechanisms based on the greedy method as well as upper conﬁdence bound (UCB). We extend these algorithms by incorporating state and action space aggregation to tackle stateaction
space explosion in the MDP. We also develop a cross entropy based method that incorpo
rates policy parameterization in order to ﬁnd near optimal energy sharing policies. Through
simulations, we show that our algorithms yield energy sharing policies that outperform the
heuristic greedy method.
