James Arthur Brown was born at Cautley, in the Gatineau Valley in Quebec, to Andrew and Caroline (Pritchard) Brown. He came to the Hocsall (Ecstall) River from Georgetown where he had been a partner in the Georgetown Mill. He purchased 160 acres for the construction of Brown's Mill and started building the sawmill in 1903 with the help of employees Bill Shand and Bill Brooks on Brown's Island. Brown's Falls and the water-powered mill is located 10 miles up the Ecstall River from Port Essington. The mill was completed in 1906 and operated under the name of J.A. Brown Mill Ltd. It supplied lumber to communities in the area and also manufactured boxes for the export of fish. When James Brown died on August 21, 1938, his stepson Jim Donaldson and sister Ada Brown Blyth took over management of the lumber mill.