Asbestos mainly affects the lungs in any of these ways:
Asbestos fibres penetrate through the lung tissue, causing a thickening of the outer covering of the lung, effectively squeezing the life out of the lung. Pleural thickening is not fatal but is very debilitating.
Asbestos fibres cause scarring and blistering inside the lungs, reducing lung capacity. Asbestosis is often fatal.
Asbestos fibres affect cells within the lung causing them to mutate into cancer cells. These cells then multiply causing tumours. Lung cancer is nearly always fatal.
Asbestos fibres penetrate through the wall of the lung causing a cancerous covering of the lung. Mesothelioma is always fatal.
Because the fibres are so small, it takes a long time for the damage to build up to the point where you notice it. Generally this can be anywhere from 15 to 60 years, depending on the type and duration of exposure.
Smoking also affects your chances of getting ill from asbestos. If you smoke and you are significantly exposed to asbestos, you are around 50 times more likely to suffer from long term health effects.