配置日志记录#
Flower 日志记录器会跟踪联邦学习工作负载中发生的所有核心事件。它默认按照标准信息格式提供信息:
DEFAULT_FORMATTER = logging.Formatter(
"%(levelname)s %(name)s %(asctime)s | %(filename)s:%(lineno)d | %(message)s"
)
相关信息包括:日志信息级别(例如 INFO
、DEBUG
)、时间戳、日志记录的行以及日志信息本身。这样,日志记录器通常会在终端上显示如下信息:
...
INFO flwr 2023-07-15 15:32:30,935 | server.py:125 | fit progress: (3, 392.5575705766678, {'accuracy': 0.2898}, 13.781953627998519)
DEBUG flwr 2023-07-15 15:32:30,935 | server.py:173 | evaluate_round 3: strategy sampled 25 clients (out of 100)
DEBUG flwr 2023-07-15 15:32:31,388 | server.py:187 | evaluate_round 3 received 25 results and 0 failures
DEBUG flwr 2023-07-15 15:32:31,388 | server.py:222 | fit_round 4: strategy sampled 10 clients (out of 100)
DEBUG flwr 2023-07-15 15:32:32,429 | server.py:236 | fit_round 4 received 10 results and 0 failures
INFO flwr 2023-07-15 15:32:33,516 | server.py:125 | fit progress: (4, 370.3378576040268, {'accuracy': 0.3294}, 16.36216809399957)
DEBUG flwr 2023-07-15 15:32:33,516 | server.py:173 | evaluate_round 4: strategy sampled 25 clients (out of 100)
DEBUG flwr 2023-07-15 15:32:33,966 | server.py:187 | evaluate_round 4 received 25 results and 0 failures
DEBUG flwr 2023-07-15 15:32:33,966 | server.py:222 | fit_round 5: strategy sampled 10 clients (out of 100)
DEBUG flwr 2023-07-15 15:32:34,997 | server.py:236 | fit_round 5 received 10 results and 0 failures
INFO flwr 2023-07-15 15:32:36,118 | server.py:125 | fit progress: (5, 358.6936808824539, {'accuracy': 0.3467}, 18.964264554999318)
...
将日志保存到文件#
默认情况下,Flower 日志会输出到启动联邦学习工作负载的终端。这既适用于基于 gRPC 的联邦学习(即执行 fl.server.start_server
时),也适用于使用 VirtualClientEngine
时(即执行 fl.simulation.start_simulation
时)。在某些情况下,您可能希望将此日志保存到磁盘。为此,您可以调用 fl.common.logger.configure() 函数。例如:
import flwr as fl
...
# in your main file and before launching your experiment
# add an identifier to your logger
# then specify the name of the file where the log should be outputted to
fl.common.logger.configure(identifier="myFlowerExperiment", filename="log.txt")
# then start your workload
fl.simulation.start_simulation(...) # or fl.server.start_server(...)
通过上述操作,Flower 会将您在终端上看到的日志记录到 log.txt
。该文件将创建在运行代码的同一目录下。如果我们检查一下,就会发现上面的日志也被记录了下来,但每一行都以 identifier
作为前缀:
...
myFlowerExperiment | INFO flwr 2023-07-15 15:32:30,935 | server.py:125 | fit progress: (3, 392.5575705766678, {'accuracy': 0.2898}, 13.781953627998519)
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:30,935 | server.py:173 | evaluate_round 3: strategy sampled 25 clients (out of 100)
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:31,388 | server.py:187 | evaluate_round 3 received 25 results and 0 failures
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:31,388 | server.py:222 | fit_round 4: strategy sampled 10 clients (out of 100)
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:32,429 | server.py:236 | fit_round 4 received 10 results and 0 failures
myFlowerExperiment | INFO flwr 2023-07-15 15:32:33,516 | server.py:125 | fit progress: (4, 370.3378576040268, {'accuracy': 0.3294}, 16.36216809399957)
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:33,516 | server.py:173 | evaluate_round 4: strategy sampled 25 clients (out of 100)
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:33,966 | server.py:187 | evaluate_round 4 received 25 results and 0 failures
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:33,966 | server.py:222 | fit_round 5: strategy sampled 10 clients (out of 100)
myFlowerExperiment | DEBUG flwr 2023-07-15 15:32:34,997 | server.py:236 | fit_round 5 received 10 results and 0 failures
myFlowerExperiment | INFO flwr 2023-07-15 15:32:36,118 | server.py:125 | fit progress: (5, 358.6936808824539, {'accuracy': 0.3467}, 18.964264554999318)
...
记录自己的信息#
您可以通过添加更多与应用程序相关的信息来扩展 Flower 日志记录器默认显示的信息。您可以通过以下方法轻松实现这一目标。
# in the python file you want to add custom messages to the Flower log
from logging import INFO, DEBUG
from flwr.common.logger import log
# For example, let's say you want to add to the log some info about the training on your client for debugging purposes
class FlowerClient(fl.client.NumPyClient):
def __init__(self, cid: int ...):
self.cid = cid
self.net = ...
...
def fit(self, parameters, config):
log(INFO, f"Printing a custom INFO message at the start of fit() :)")
set_params(self.net, parameters)
log(DEBUG, f"Client {self.cid} is doing fit() with config: {config}")
...
这样,除默认信息外,您的日志记录器还将显示由客户引入的信息,如上文所述。
...
INFO flwr 2023-07-15 16:18:21,726 | server.py:89 | Initializing global parameters
INFO flwr 2023-07-15 16:18:21,726 | server.py:276 | Requesting initial parameters from one random client
INFO flwr 2023-07-15 16:18:22,511 | server.py:280 | Received initial parameters from one random client
INFO flwr 2023-07-15 16:18:22,511 | server.py:91 | Evaluating initial parameters
INFO flwr 2023-07-15 16:18:25,200 | server.py:94 | initial parameters (loss, other metrics): 461.2934241294861, {'accuracy': 0.0998}
INFO flwr 2023-07-15 16:18:25,200 | server.py:104 | FL starting
DEBUG flwr 2023-07-15 16:18:25,200 | server.py:222 | fit_round 1: strategy sampled 10 clients (out of 100)
INFO flwr 2023-07-15 16:18:26,391 | main.py:64 | Printing a custom INFO message :)
DEBUG flwr 2023-07-15 16:18:26,391 | main.py:63 | Client 44 is doing fit() with config: {'epochs': 5, 'batch_size': 64}
INFO flwr 2023-07-15 16:18:26,391 | main.py:64 | Printing a custom INFO message :)
DEBUG flwr 2023-07-15 16:18:28,464 | main.py:63 | Client 99 is doing fit() with config: {'epochs': 5, 'batch_size': 64}
INFO flwr 2023-07-15 16:18:28,465 | main.py:64 | Printing a custom INFO message :)
DEBUG flwr 2023-07-15 16:18:28,519 | main.py:63 | Client 67 is doing fit() with config: {'epochs': 5, 'batch_size': 64}
INFO flwr 2023-07-15 16:18:28,519 | main.py:64 | Printing a custom INFO message :)
DEBUG flwr 2023-07-15 16:18:28,615 | main.py:63 | Client 11 is doing fit() with config: {'epochs': 5, 'batch_size': 64}
INFO flwr 2023-07-15 16:18:28,615 | main.py:64 | Printing a custom INFO message :)
DEBUG flwr 2023-07-15 16:18:28,617 | main.py:63 | Client 13 is doing fit() with config: {'epochs': 5, 'batch_size': 64}
...
登录远程服务#
此外,fl.common.logger.configure`函数还允许指定主机,通过本地 Python :code:`logging.handler.HTTPHandler
,向该主机推送日志(通过 POST
)。在基于 gRPC
的联邦学习工作负载中,这是一个特别有用的功能,否则从所有实体(即服务器和客户端)收集日志可能会很麻烦。请注意,在 Flower 模拟器中,服务器会自动显示所有日志。如果希望在其他地方备份或分析日志,仍可指定 HTTPHandler
。