FedScale | A scalable and extensible federated learning engine and benchmark.

Aggregation Optimizations

This page reports benchmarking results of various aggregation optimizations, including server-side optimizers.

The bold method name indicates that the implementation is official (by the author of the original paper).

Please consider submitting your results!

OpenImage (Image Classification)

Rank	Method	Model	Test Accuracy	Training Rounds	FedScale Runtime(h)	Contact	References	Date
1	FedYoGi	MobileNet-V2	0.7525 ± 0.0007	1930	258	FedScale Team	Paper, Code	Feb 13, 2022
2	FedYoGi	ShuffleNet-V2	0.7404 ± 0.001	1872	232	FedScale Team	Paper, Code	Feb 13, 2022
3	FedProx	MobileNet-V2	0.7034 ± 0.0013	1680	223	FedScale Team	Paper, Code	Feb 13, 2022
4	FedAvg	ShuffleNet-V2	0.7027 ± 0.0011	2070	273	FedScale Team	Paper, Code	Feb 13, 2022
5	FedAvg	MobileNet-V2	0.7009 ± 0.0008	2190	291	FedScale Team	Paper, Code	Feb 13, 2022
6	FedProx	ShuffleNet-V2	0.6954 ± 0.0015	1665	221	FedScale Team	Paper, Code	Feb 13, 2022

FEMNIST (Image Classification)

Rank	Method	Model	Test Accuracy	Training Rounds	FedScale Runtime(h)	Contact	References	Date
1	FedAvg	ResNet-18	0.7850 ± 0.0009	-	-	FedScale Team	Paper, Code	Feb 13, 2022
2	FedProx	ResNet-18	0.7840 ± 0.0012	-	-	FedScale Team	Paper, Code	Feb 13, 2022
3	FedYoGi	MobileNet-V2	0.7630 ± 0.0015	-	-	FedScale Team	Paper, Code	Feb 13, 2022

Google Speech (Speech Recognition)

Rank	Method	Model	Test Accuracy	Training Rounds	FedScale Runtime(h)	Contact	References	Date
1	FedAvg	ResNet-34	0.6337 ± 0.0014	480	108	FedScale Team	Paper, Code	Feb 13, 2022
2	FedProx	ResNet-34	0.6325 ± 0.0011	555	125	FedScale Team	Paper, Code	Feb 13, 2022
3	FedYoGi	ResNet-34	0.6267 ± 0.0023	600	133	FedScale Team	Paper, Code	Feb 13, 2022