Remark. Parameter sharing and width tradeoffs [ftip-00KJ]
Remark. Parameter sharing and width tradeoffs [ftip-00KJ]
Xue et al. report that depth sharing can reduce trainable parameters while limiting modeling capacity, and study width and mixture-of-experts remedies in Go Wider Instead of Deeper[xue2021wider]. The comparison is empirical and task-specific; it does not identify a depth-independent ceiling.