The concept of a point dipole is potentially ambiguous due to inherent singularity of electro-magnetic fields at its location. We discuss this concept from several points of view. First, we consider a point dipole as a singular point in space whose sole ability is to be polarized due to the external electric field. We introduce the source Green's dyadic that provides a unified albeit empiric description of the contribution of the dipole to the electromagnetic properties of the whole space. We argue that this is the most complete, concise, and unambiguous definition of a point dipole and its polarizability. Next, we revisit classic expressions for absorption and emission power by integration of Poynting vector over a surface enclosing the point dipole, thereby avoiding the singularity. We also consider the energy balance between the fluctuating dipole moment and the medium (thermal bath) to derive the fluctuation-dissipation theorem in terms of fluctuating dipole moment. This solves the long-standing controversy in the literature. Second, the same results can be obtained for a very small homogeneous sphere, in which the internal field is known to be constant. This leads to unambiguous microscopic definition of the particle dipole moment and polarizability in terms of its size and refractive index. Third, and most interestingly, we generalize this microscopic description to small particles of arbitrary shape. Both bare (electrostatic) and dressed (corrected) polarizabilities are defined as double integrals of the corresponding dyadic transition operator over the particle's volume.