My guess is that you are making a mistake in how you use the Functional API. Calling a layer function returns you a function and then you call that function with the actual input tensor. My guess is you only have one level of call.
The documentation in the notebooks about how to use the Sequential and Functional APIs is pretty minimal. I would recommend reading this helpful thread on the forums which gives a much nicer explanation of how to use these TF features.