TensorFlow and Keras

TensorFlow provides tensor execution, automatic differentiation, saved models, and deployment tooling; Keras is its high-level model-building API. A Keras Dense layer still implements the same affine-plus-activation mechanism from neural network fundamentals, but the framework organizes training through compile, fit, losses, metrics, callbacks, and export paths. PyTorch tends to make the custom training loop more visible.

A dense classifier

A dense classifier computes

$$z=xW+b,p=\mathrm{softmax}(z),$$

Here $x$ is a batch of input features, $W$ and $b$ are learned dense-layer parameters, $z$ is the vector of class logits, and $p$ is the resulting probability vector after softmax. The framework hides some plumbing, but this is still the computation being differentiated.

then minimizes a loss, often cross-entropy:

$$L=-\log p_y.$$

The target class is $y$, so $p_y$ is the predicted probability assigned to the correct class. Minimizing $L$ increases that probability.

Keras compile(optimizer=..., loss=...) binds that objective to an optimizer; fit() repeatedly batches data, records gradients, and applies updates. Lower-level TensorFlow uses tf.GradientTape for custom loops.

Worked example

TensorFlow is not installed in this workspace, so this runnable NumPy snippet shows the exact dense-layer and softmax computation that a one-layer Keras classifier would perform.

import numpy as np
 
np.random.seed(15)
x = np.array([[1.0, -2.0]])
W = np.random.normal(size=(2, 3))
b = np.array([0.1, 0.0, -0.1])
logits = x @ W + b
probs = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)
print("dense_logits", np.round(logits, 3).tolist())
print("softmax_probs", np.round(probs, 3).tolist())
print("predicted_class", probs.argmax(axis=1).tolist())

Observed output:

dense_logits [[0.791, -0.132, 3.271]]
softmax_probs [[0.075, 0.03, 0.895]]
predicted_class [2]

The highest logit dominates the softmax distribution, so the predicted class is index 2. Keras would wrap this same computation in layer objects and training callbacks.

Caveats

High-level APIs reduce boilerplate but can hide defaults: loss reduction, metric state, training/evaluation mode, and callback side effects matter. Serialization format and serving target should influence design early if the model must run outside Python.

References

• TensorFlow guide: Keras, the high-level API for TensorFlow
• PyTorch documentation: Autograd mechanics

Nav

Section — Deep Learning

← PyTorch Mixed Precision →