Pressure is mounting on the Albanese government as new polling shows Australians feeling the pinch are losing faith in Labor.