Original Blog: Doi Tech Team
Link: https://blog.doiduoyi.com/authors/1584446358138
Original Intent: Record the learning experiences of the excellent Doi Tech Team
This article is based on PaddlePaddle 0.13.0 and Python 2.7
For Fluid version usage, please refer to the author’s new tutorial series: 《PaddlePaddle From Beginner to炼丹》
Introduction¶
Fluid was introduced in PaddlePaddle 0.11.0. It is designed to allow users to execute programs like PyTorch and TensorFlow Eager Execution. In these systems, there is no longer a concept of “model”; applications no longer contain symbolic descriptions of Operator graphs or layers, but instead describe the training or inference process like general programs.
The key difference between Fluid and PyTorch/Eager Execution is that Fluid does not rely on Python’s control flow (e.g., if-else-then or for loops). Instead, it provides control flow implemented in C++ with corresponding Python interfaces via the with syntax. For example, the code snippet we’ll use:
with fluid.program_guard(inference_program):
test_accuracy = fluid.evaluator.Accuracy(input=out, label=label)
test_target = [avg_cost] + test_accuracy.metrics + test_accuracy.states
inference_program = fluid.io.get_inference_program(test_target)
In the Fluid version, the trainer is replaced by a C++ class Executor (similar to an interpreter) to run Fluid programs for training and inference:
loss, acc = exe.run(fluid.default_main_program(),
feed=feeder.feed(data),
fetch_list=[avg_cost] + accuracy.metrics)
Since we haven’t used the Fluid version before, we’ll explore its changes by comparing it with previous implementations.
Training the Model¶
1. Defining the Neural Network¶
We use the familiar VGG16 model (previously used for CIFAR color image recognition). For consistency, we also use the CIFAR10 dataset. Below are the Paddle 1 and Fluid versions of VGG16 for comparison:
Paddle 1 Version
def vgg_bn_drop(input, class_dim):
# Define convolutional block
def conv_block(ipt, num_filter, groups, dropouts, num_channels=None):
return paddle.networks.img_conv_group(
input=ipt,
num_channels=num_channels,
pool_size=2,
pool_stride=2,
conv_num_filter=[num_filter] * groups,
conv_filter_size=3,
conv_act=paddle.activation.Relu(),
conv_with_batchnorm=True,
conv_batchnorm_drop_rate=dropouts,
pool_type=paddle.pooling.Max())
conv1 = conv_block(input, 64, 2, [0.3, 0], 3)
conv2 = conv_block(conv1, 128, 2, [0.4, 0])
conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0])
conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])
conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])
drop = paddle.layer.dropout(input=conv5, dropout_rate=0.5)
fc1 = paddle.layer.fc(input=drop, size=512, act=paddle.activation.Linear())
bn = paddle.layer.batch_norm(input=fc1,
act=paddle.activation.Relu(),
layer_attr=paddle.attr.Extra(drop_rate=0.5))
fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear())
predict = paddle.layer.fc(input=fc2,
size=class_dim,
act=paddle.activation.Softmax())
return predict
Fluid Version
def vgg16_bn_drop(input):
# Define convolutional block
def conv_block(input, num_filter, groups, dropouts):
return fluid.nets.img_conv_group(
input=input,
pool_size=2,
pool_stride=2,
conv_num_filter=[num_filter] * groups,
conv_filter_size=3,
conv_act='relu',
conv_with_batchnorm=True,
conv_batchnorm_drop_rate=dropouts,
pool_type='max')
conv1 = conv_block(input, 64, 2, [0.3, 0])
conv2 = conv_block(conv1, 128, 2, [0.4, 0])
conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0])
conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])
conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])
drop = fluid.layers.dropout(x=conv5, dropout_prob=0.5)
fc1 = fluid.layers.fc(input=drop, size=512, act=None)
bn = fluid.layers.batch_norm(input=fc1, act='relu')
drop2 = fluid.layers.dropout(x=bn, dropout_prob=0.5)
fc2 = fluid.layers.fc(input=drop2, size=512, act=None)
# Classifier with Softmax
predict = fluid.layers.fc(
input=fc2,
size=class_dim,
act='softmax',
param_attr=ParamAttr(name="param1", initializer=NormalInitializer()))
return predict
Using the Network
class_dim = 10 # CIFAR10 has 10 classes
predict = vgg16_bn_drop(image, class_dim)
2. Defining Data¶
In Fluid, data is defined with explicit shape and type parameters:
Fluid Version
image_shape = [3, 32, 32] # [channels, height, width]
image = fluid.layers.data(name='image', shape=image_shape, dtype='float32')
label = fluid.layers.data(name='label', shape=[1], dtype='int64')
Paddle 1 Version
image = paddle.layer.data(name="image",
type=paddle.data_type.dense_vector(datadim))
label = paddle.layer.data(name="label",
type=paddle.data_type.integer_value(type_size))
3. Defining Batch Average Accuracy¶
Fluid introduces batch_acc to calculate average accuracy during training/inference, defined before the optimization step:
batch_size = fluid.layers.create_tensor(dtype='int64')
batch_acc = fluid.layers.accuracy(input=predict, label=label, total=batch_size)
4. Defining the Inference Program¶
The inference program is cloned from the main program for testing:
inference_program = fluid.default_main_program().clone(for_test=True)
5. Defining the Optimization Method¶
Fluid consolidates learning_rate logic in one place:
Fluid Version
optimizer = fluid.optimizer.Momentum(
learning_rate=fluid.layers.exponential_decay(
learning_rate=learning_rate,
decay_steps=40000,
decay_rate=0.1,
staircase=True),
momentum=0.9,
regularization=fluid.regularizer.L2Decay(0.0005), )
opts = optimizer.minimize(loss)
Paddle 1 Version
momentum_optimizer = paddle.optimizer.Momentum(
momentum=0.9,
regularization=paddle.optimizer.L2Regularization(rate=0.0002 * 128),
learning_rate=0.1 / 128.0,
learning_rate_decay_a=0.1,
learning_rate_decay_b=50000 * 100,
learning_rate_schedule='discexp')
6. Training and Testing¶
Executor Initialization
Fluid uses Executor instead of trainer:
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program()) # Initialize parameters
Data Loading
train_reader = paddle.batch(
paddle.dataset.cifar.train10(), batch_size=BATCH_SIZE)
test_reader = paddle.batch(
paddle.dataset.cifar.test10(), batch_size=BATCH_SIZE)
feeder = fluid.DataFeeder(place=place, feed_list=[image, label])
Training Loop
Fluid uses a Python loop for epochs, simplifying training control:
accuracy = fluid.average.WeightedAverage()
test_accuracy = fluid.average.WeightedAverage()
for pass_id in range(num_passes):
accuracy.reset()
# Training batches
for batch_id, data in enumerate(train_reader()):
loss, acc, weight = exe.run(
fluid.default_main_program(),
feed=feeder.feed(data),
fetch_list=[avg_cost, batch_acc, batch_size])
accuracy.add(value=acc, weight=weight)
print(f"Pass {pass_id}, batch {batch_id}, loss {loss[0]}, acc {acc[0]}")
# Testing batches
test_accuracy.reset()
for data in test_reader():
loss, acc, weight = exe.run(
inference_program,
feed=feeder.feed(data),
fetch_list=[avg_cost, batch_acc, batch_size])
test_accuracy.add(value=acc, weight=weight)
pass_acc = accuracy.eval()
test_pass_acc = test_accuracy.eval()
print(f"End pass {pass_id}, train_acc {pass_acc}, test_acc {test_pass_acc}")
# Save model
model_path = os.path.join(model_save_dir, str(pass_id))
fluid.io.save_inference_model(model_path, ['image'], [predict], exe)
7. Saving the Inference Model¶
Fluid uses save_inference_model for deployment:
fluid.io.save_inference_model(model_path, ['image'], [predict], exe)
Inference¶
1. Executor Initialization¶
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
2. Loading the Trained Model¶
[inference_program, feed_target_names, fetch_targets] = fluid.io.load_inference_model(save_dirname, exe)
3. Preprocessing the Image¶
img = Image.open(image_file)
img = img.resize((32, 32), Image.ANTIALIAS)
test_data = np.array(img).astype("float32")
test_data = np.transpose(test_data, (2, 0, 1)) # HWC → CHW
test_data = test_data[np.newaxis, :] / 255.0
4. Running Inference¶
results = exe.run(inference_program,
feed={feed_target_names[0]: test_data},
fetch_list=fetch_targets)
results = np.argsort(-results[0])
print(f"Predicted label: {results[0][0]}")
Project Code¶
GitHub Repository: https://github.com/yeyupiaoling/LearnPaddle
References¶
Previous Chapter: 《我的PaddlePaddle学习之路》笔记十——自定义图像数据集实现目标检测¶
Next Chapter: 《我的PaddlePaddle学习之路》笔记十二——VisualDL可视化工具的使用¶
Note: Some terms are retained in Chinese for technical accuracy (e.g., “炼丹” = “Alchemy”, a metaphor for model training).