Knowledge of swelling properties of shale-oil formations as well as the effects of various drilling fluids, including both water-based and oil-based muds (OBMs), which are of critical importance to wellbore-stability problems, requires a better understanding of shale-oil properties. Drilling through shale-oil formations is highly problematic and imposes substantial costs to the operators owing to wellbore-stability problems. These problems include, but are not limited to, tight holes, stuck pipe, fishing, sidetracking, and well abandonment. To more efficiently and effectively drill through these formations, we should better understand their properties. Only few experiments have been conducted on shale-oil samples to better understand their properties. Most experiments performed thus far were run on common shale core samples, which are significantly different from shale-oil samples in the matter of mineralogy and mechanical properties. Subsequently, the results of those experiments cannot be equally applied to shale-oil and shale-gas formations. In this study, we first determined the mineralogy of shale-oil core samples from the Eagle Ford field and then investigated the swelling properties and cation exchange capacity (CEC) of the core samples in the laboratory. Experiments have been conducted with the samples partially submerged in distilled water, potassium-chloride (7% KCl) brine, and OBM. Several experiments have been performed using strain gages to measure lateral, axial, and diagonal swelling in both submerged and non-submerged areas. We also performed unconfined compressive strength (UCS) tests to investigate the effect of the invasion of various drilling fluids on the compressive strength of the core samples. The experimental setup was modified to accommodate five linearly variable displacement transducers (LVDTs) to measure Young's Modulus (E) and Poisson's ratio (v). Various experiments were run to quantify the effect of temperature on the rock compressive strength, E, and v. Experiments have shown a distinct change in the mechanical properties of the rock. The results demonstrate that the swelling properties and CEC of the shale-oil core samples are different from the common shale core samples. This study proposes the quantification of the shale/fluid properties, the interaction, and the effects of different fluids on rock properties in unconventional reservoirs. This paper presents and documents the differences in the swelling properties between conventional and unconventional shale. The results of the study will help us to more precisely understand unconventional shale-oil rock properties and can be used to design a more effective drilling fluid for field applications, as well as more accurately predict the mechanisms of formation failure.