At the forefront of neuroimaging is the understanding of the functional architecture of the human brain. In most applications functional networks are assumed to be stationary, resulting in a single network estimated for the entire time course. However recent results suggest that the connectivity between brain regions is highly non-stationary even at rest. As a result, there is a need for new brain imaging methodologies that comprehensively account for the dynamic nature of functional networks. In this work we propose the Smooth Incremental Graphical Lasso Estimation (SINGLE) algorithm which estimates dynamic brain networks from fMRI data. We apply the proposed algorithm to functional MRI data from 24 healthy patients performing a Choice Reaction Task to demonstrate the dynamic changes in network structure that accompany a simple but attentionally demanding cognitive task. Using graph theoretic measures we show that the properties of the Right Inferior Frontal Gyrus and the Right Inferior Parietal lobe dynamically change with the task. These regions are frequently reported as playing an important role in cognitive control. Our results suggest that both these regions play a key role in the attention and executive function during cognitively demanding tasks and may be fundamental in regulating the balance between other brain regions.