Show simple item record

dc.contributor.authorKhaleghian, Salman
dc.contributor.authorUllah, Habib
dc.contributor.authorJohnsen, Einar Broch
dc.contributor.authorAndersen, Anders
dc.contributor.authorMarinoni, Andrea
dc.date.accessioned2022-11-24T09:29:20Z
dc.date.available2022-11-24T09:29:20Z
dc.date.issued2022-08-08
dc.description.abstractWe propose a novel and adaptive feature space distillation method (AFSD) to reduce the communication overhead among distributed computers. The proposed method improves the Codistillation process by supporting longer update interval rates. AFSD performs knowledge distillates across the models infrequently and provides flexibility to the models in terms of exploring diverse variations in the training process. We perform knowledge distillation in terms of sharing the feature space instead of output only. Therefore, we also propose a new loss function for the Codistillation technique in AFSD. Using the feature space leads to more efficient knowledge transfer between models with a longer update interval rates. In our method, the models can achieve the same accuracy as Allreduce and Codistillation with fewer epochs.en_US
dc.identifier.citationKhaleghian S, Ullah H, Johnsen E. B., Andersen A, Marinoni A. AFSD: Adaptive Feature Space Distillation for Distributed Deep Learning. IEEE Access. 2022;10:84569-84578en_US
dc.identifier.cristinIDFRIDAID 2066915
dc.identifier.doi10.1109/ACCESS.2022.3197646
dc.identifier.issn2169-3536
dc.identifier.urihttps://hdl.handle.net/10037/27512
dc.language.isoengen_US
dc.publisherIEEEen_US
dc.relation.ispartofKhaleghian, S. (2022). Scalable computing for earth observation - Application on Sea Ice analysis. (Doctoral thesis). <a href=https://hdl.handle.net/10037/27513>https://hdl.handle.net/10037/27513</a>.
dc.relation.journalIEEE Access
dc.relation.projectIDNorges forskningsråd: 237898en_US
dc.relation.projectIDinfo:eu-repo/grantAgreement/EC/H2020/825258/EU/From Copernicus Big Data to Extreme Earth Analytics/ExtremeEarth/en_US
dc.rights.accessRightsopenAccessen_US
dc.rights.holderCopyright 2022 The Author(s)en_US
dc.rights.urihttps://creativecommons.org/licenses/by/4.0en_US
dc.rightsAttribution 4.0 International (CC BY 4.0)en_US
dc.titleAFSD: Adaptive Feature Space Distillation for Distributed Deep Learningen_US
dc.type.versionpublishedVersionen_US
dc.typeJournal articleen_US
dc.typeTidsskriftartikkelen_US
dc.typePeer revieweden_US


File(s) in this item

Thumbnail

This item appears in the following collection(s)

Show simple item record

Attribution 4.0 International (CC BY 4.0)
Except where otherwise noted, this item's license is described as Attribution 4.0 International (CC BY 4.0)