首页 > 搜索 > stardiscover算法,Paper:2020年3月30日何恺明团队最新算法RegNet—来自Facebook AI研究院《Designing Network Design Spaces》的翻译与解读

stardiscover算法,Paper:2020年3月30日何恺明团队最新算法RegNet—来自Facebook AI研究院《Designing Network Design Spaces》的翻译与解读

互联网 2020-10-28 13:08:33
在线算命,八字测算命理

Figure 1. Design space design. We propose to design network design spaces, where a design space is a parametrized set of possible model architectures. Design space design is akin to manual network design, but elevated to the population level. In each step of our process the input is an initial design space and the output is a refined design space of simpler or better models. Following [21], we characterize the quality of a design space by sampling models and inspecting their error distribution. For example, in the figure above we start with an initial design space A and apply two refinement steps to yield design spaces B then C. In this case C ⊆ B ⊆A (left), and the error distributions are strictly improving from A to B to C (right). The hope is that design principles that apply to model populations are more likely to be robust and generalize.

While manual network design has led to large advances, finding well-optimized networks manually can be challenging, especially as the number of design choices increases. A popular approach to address this limitation is neural architecture search (NAS). Given a fixed search space of possible networks, NAS automatically finds a good model within the search space. Recently, NAS has received a lot of attention and shown excellent results [34, 18, 29].

Despite the effectiveness of NAS, the paradigm has limitations. The outcome of the search is a single network instance tuned to a specific setting (e.g., hardware platform). This is sufficient in some cases; however, it does not enable discovery of network design principles that deepen our understanding and allow us to generalize to new settings. In particular, our aim is to find simple models that are easy to understand, build upon, and generalize.

图1所示。设计空间设计。我们建议设计网络设计空间,其中设计空间是可能的模型架构的参数化集合。设计空间设计类似于人工网络设计,但提升到了人口层面。在我们流程的每个步骤中,输入是初始设计空间,输出是更简单或更好模型的精细化设计空间。在[21]之后,我们通过采样模型并检查它们的误差分布来描述设计空间的质量。例如,在上图中我们从最初的设计空间和应用两个改进措施产量设计空间B C。在这种情况下C⊆B⊆(左),和误差分布严格改善从A到B C(右)。希望适用于模型总体的设计原则更有可能是健壮的和一般化的。

虽然手动网络设计已经取得了很大的进展,但是手动找到优化良好的网络可能是一项挑战,特别是在设计选择的数量增加的情况下。解决这一限制的一种流行方法是神经架构搜索(NAS)。给定一个可能的网络的固定搜索空间,NAS会自动在搜索空间中找到一个好的模型。近年来,NAS受到了广泛的关注,并取得了良好的研究成果[34,18,29]。

尽管NAS有效,但这种范式也有局限性。搜索的结果是将单个网络实例调优到特定的设置(例如,硬件平台)。在某些情况下,这就足够了;然而,它并不能帮助我们发现网络设计原则,从而加深我们的理解,并使我们能够归纳出新的设置。特别是,我们的目标是找到易于理解、构建和泛化的简单模型。

免责声明:非本网注明原创的信息,皆为程序自动获取互联网,目的在于传递更多信息,并不代表本网赞同其观点和对其真实性负责;如此页面有侵犯到您的权益,请给站长发送邮件,并提供相关证明(版权证明、身份证正反面、侵权链接),站长将在收到邮件12小时内删除。

相关阅读

一周热门

查看更多