Skip to content

Assignment of column via .loc for numpy non-ns datetimes #27928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Sep 13, 2019
Merged

Assignment of column via .loc for numpy non-ns datetimes #27928

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
ed9d7f0
Fix issue 27395
inmoonlight Aug 15, 2019
49471e7
Fix lint error
inmoonlight Aug 15, 2019
9efb169
shoot numpy-dev future warning
inmoonlight Aug 15, 2019
41df89d
Reflect reviews
inmoonlight Aug 15, 2019
6df540c
Fix ci error
inmoonlight Aug 15, 2019
98a5061
Fix dtype match except [ps]
inmoonlight Aug 16, 2019
299a5b1
Merge remote-tracking branch 'upstream/master' into inmoonlight-hotfi…
TomAugspurger Aug 21, 2019
cf18fb9
Reflect reviews
inmoonlight Aug 21, 2019
09ab203
Fix conflicts
inmoonlight Aug 21, 2019
93c2f85
Fix CI error
inmoonlight Aug 29, 2019
ad771f5
Fix conficts
inmoonlight Aug 31, 2019
35009b7
Fix CI error
inmoonlight Aug 31, 2019
6a48ca6
Fix CI error
inmoonlight Aug 31, 2019
063cc37
Merge branch 'master' into hotfix/27395/loc-assignment
inmoonlight Sep 12, 2019
d64a2d8
Update whatsnew
inmoonlight Sep 12, 2019
86fd1c1
Add comments
inmoonlight Sep 12, 2019
2ddbc01
Merge branch 'master' into hotfix/27395/loc-assignment
inmoonlight Sep 13, 2019
48e454a
Fix CI error
inmoonlight Sep 13, 2019
c8eb91f
Fix CI error
inmoonlight Sep 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.25.1.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Categorical
Datetimelike
^^^^^^^^^^^^
- Bug in :func:`to_datetime` where passing a timezone-naive :class:`DatetimeArray` or :class:`DatetimeIndex` and ``utc=True`` would incorrectly return a timezone-naive result (:issue:`27733`)
-
- Bug in :func:`maybe_cast_to_datetime` where converting a `np.datetime64` to `datetime64[D]` raise `TypeError` (:issue: `27395`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a private function, you would have to indicate the user facing experience here instead (e.g. the assignment)

-
-

Expand Down
4 changes: 2 additions & 2 deletions pandas/core/dtypes/cast.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1026,7 +1026,7 @@ def maybe_cast_to_datetime(value, dtype, errors="raise"):
)

if is_datetime64 and not is_dtype_equal(dtype, _NS_DTYPE):
if dtype.name in ("datetime64", "datetime64[ns]"):
if dtype.name in ("datetime64", "datetime64[ns]", "datetime64[D]"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just use dtype.kind == 'M'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since pandas doesn't support to convert datetime[ps], I used (dtype.kind == 'M') and (dtype.name != 'datetime64[ps]')

if dtype.name == "datetime64":
raise ValueError(msg.format(dtype=dtype.name))
dtype = _NS_DTYPE
Expand All @@ -1044,7 +1044,7 @@ def maybe_cast_to_datetime(value, dtype, errors="raise"):
value = [value]

elif is_timedelta64 and not is_dtype_equal(dtype, _TD_DTYPE):
if dtype.name in ("timedelta64", "timedelta64[ns]"):
if dtype.name in ("timedelta64", "timedelta64[ns]", "timedelta64[D]"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dtype.kind == 'm'

Copy link
Contributor Author

@inmoonlight inmoonlight Aug 16, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since pandas doesn't support to convert timedelta[ps], I used (dtype.kind == 'm') and (dtype.name != 'timedelta64[ps]')

if dtype.name == "timedelta64":
raise ValueError(msg.format(dtype=dtype.name))
dtype = _TD_DTYPE
Expand Down
12 changes: 12 additions & 0 deletions pandas/tests/dtypes/cast/test_infer_dtype.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
cast_scalar_to_array,
infer_dtype_from_array,
infer_dtype_from_scalar,
maybe_cast_to_datetime,
)
from pandas.core.dtypes.common import is_dtype_equal

Expand Down Expand Up @@ -169,3 +170,14 @@ def test_cast_scalar_to_array(obj, dtype):

arr = cast_scalar_to_array(shape, obj, dtype=dtype)
tm.assert_numpy_array_equal(arr, exp)


@pytest.mark.parametrize(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its ok to test this i guess, but would need to parameterize on datetime64[D], ns, datetime64 etc and check the result

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. I made all possible datetime formats except [ps] (as mentioned above)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @jreback's comment was requesting testing those other frequencies here.

And then assert the result matches too.

result = maybe_cast_to_datetime(obj, dtype)
expected = pd.Timestamp(...)  # right?
assert result == expected

"obj,dtype",
[
(np.datetime64("2017-01-01 01:00:00"), "datetime64[D]"),
(np.datetime64("2017-01-01 02:00:00"), "datetime64[D]"),
],
)
def test_maybe_cast_to_datetime(obj, dtype):
maybe_cast_to_datetime(obj, dtype)