Intense street fighting developed as the Federals wrestled with Confederates for control. Anticipating this, Barkdale’s men placed dirt filled boxes and barrels on the streets for additional cover. Ordered to withdraw, Barksdale complied but stubbornly contested the Federal advance. Fighting behind these makeshift measures and doorways, trees, windows and anything else that offered cover, his men exacted a heavy toll on the Federal infantry.
Among Nineteenth Mass. Co. B was Sgt. Joseph Hodgkins: “[We] formed on the bank, and deployed as skirmishers through the streets. Our company moved up the street leading from the river, where we had crossed, when we received a shower of bullets from the enemy, who were posted in the houses behind the fences and wherever a shelter offered. We turned into a corner lot, and took shelter behind a fence, when we received another volley, killing one of our men and wounding another. We returned the fire, but found the Rebs. too thick and too well posted for us, so we beat a retreat to the river again.” Hawke Street was too well defended and so Sgt. Hodgkins and the rest of the Nineteenth Mass. fought from behind the house at the northwestern corner of Hawke and Sophia Street .
Nineteenth Massachusetts Lt. John Adams corroborated Sgt. Hodgkins and provided more details: “As soon as the boats touched the shore we formed by companies, and, without waiting for regimental formation, charged up the street, on reaching the main street we found that the fire came from houses in front and rear. Company B lost ten men out of thirty in less than five minutes. Other companies suffered nearly the same. We were forced to fall back to the river, deploy as skirmishers, and reached the main street through the yards and houses... Capt. Weymouth went from right to left of the line, giving instructions and urging the men forward. My squad was composed of men from companies I and A. We had reached a gate, and were doing our best to cross the street.. I had lost three men when Captain Weymouth came up. ‘Can’t you go forward, Lieutenant Adams?’ he said. My reply was, ‘It is mighty hot, captain.’ He said, ‘I guess you can,’ and started to go through the gate, when as much as a barrel of bullets came at him. He turned and said, ‘It is quite warm, lieutenant; go up through the house.’ We then entered the back door and passed upstairs to the front, Gilman Nichols of Company A was in advance. He found the door locked and burst it open with the butt of his musket. The moment it opened he fell dead, shot from a house on the other side of the street. Several others were wounded, but we held the house until dark, firing at a head whenever we saw one on the other side.”
While the Nineteenth Massachusetts fought from behind houses, the Seventh Michigan fought from behind the southeast corner house on the same intersection (the streets of Fredericksburg follow the Rappahannock River and so the first street, Sophia, runs along a NW to SE axis). The freshest regiment to land was the Twentieth Massachusetts. They stormed up Hawke Street with disastrous results. Among the waiting Confederates was Thirteenth Mississippi Pvt. William Little Davis. “One Yankee reg’t formed a pretty line and was advancing up a street-a little before dark, our boys who were laying on the ground on the next street, quickly arose and poured a deadly volley into their midst causing the greatest confusion. The Yankee officer cried to his men to charge the rebels. Our boys, one and all, cried out 'Come on!'”
The Mississippians contested the ground stubbornly and describing the Thirteenth Mississippi’s volley from the receiving end is Twentieth Massachusetts Volunteer Infantry’s Pvt. Josiah F. Murphey: “Think of it, a company of about sixty men advancing up a street with no protection whatever and two or three hundred of the enemy sheltered completely and pouring a murderous fire upon you from every window, door, and behind every fence. They would even poke their guns around the corner of the houses and fire into us at close range... We lost about forty men from our company in the space of fifty yards. In no battle of the war did we lose so many men in so short a time.” Pvt. Murphey, who was among the wounded, mentions only the casualties of his company and his brigade commander, Colonel Norman Hall, reported that they suffered ninety-seven men and officers killed and wounded within that fifty yards.
Despite sustaining heavy casualties, the Twentieth Massachusetts stormed up one block to Caroline Street and wheeling right and left, cleared Caroline of the Confederates. By now Barksdale had been ordered to withdraw but one of his officers, Twenty-first Mississippi’s Lt. Lane Brandon recognized his Harvard Law School classmate, Maj. Henry Abbott, and refused to yield ground to Abbott. Instead Lt. Brandon counterattacked and drove Abbott back. Lt. Brandon received another order to retreat and after he again refused, had to be arrested.