2020年05月
2020-5-6 周三
停止 crontab 的异常进程
有一个异常进程，使用 kill -9 杀不掉，杀掉后他自动会开一个新的 PID，资源占用 100%，初步排查是 crontab 里每月 1/16 号执行的定时任务
可以用如下命令进行重新启动 crontab
停止服务：
/etc/init.d/cron stop
启动服务：
/etc/init.d/cron start
然后异常进程就消失了，这只是临时解决一下，具体问题应该还是要看代码中的异常
2020-5-7 周四
Sentry 邮箱配置
用 QQ 域名邮箱配置了好久都不成功，换成了 126 一次就成功了
QQ 邮箱会报这个错：
我是在 docker env 里配置的，用的 tls 这样配置也可以：
SENTRY_SERVER_EMAIL = 'aaa@126.com' SENTRY_EMAIL_USER = 'aaa@126.com' SENTRY_EMAIL_PASSWORD = 'NABGTAAAAAAYACAM' SENTRY_EMAIL_HOST = 'smtp.126.com' SENTRY_EMAIL_PORT = 25 SENTRY_EMAIL_USE_TLS = true
有看到其他人不在 env 中配置也行：Sentry 搭建
2020-5-8 周五
Request 可能存在的坑
$user_ids = $request->input('user_ids', []);
上面这样，前端传 null 的时候，收到的还是 null
像下面这样写会安全一些
$user_ids = $request->input('user_ids') ?? [];
2020-5-11 周一
表达式引擎
// TODO
2020-5-12 周二
Sentry Features
Sentry 安装完成之后发现有一些特性和说好的不一样，这是因为没有配置的问题
SENTRY_FEATURES = {
# Enables user registration.
"auth:register": True,
# Enable advanced search features, like negation and wildcard matching.
"organizations:advanced-search": True,
# Enable obtaining and using API keys.
"organizations:api-keys": False,
# Enable explicit use of AND and OR in search.
"organizations:boolean-search": False,
# Enable creating organizations within sentry (if SENTRY_SINGLE_ORGANIZATION
# is not enabled).
"organizations:create": True,
# Enable the 'discover' interface.
"organizations:discover": False,
# Enable attaching arbitrary files to events.
"organizations:event-attachments": False,
# Allow organizations to configure built-in symbol sources.
"organizations:symbol-sources": True,
# Allow organizations to configure custom external symbol sources.
"organizations:custom-symbol-sources": True,
# Enable the events stream interface.
"organizations:events": False,
# Enable events v2 instead of the events stream
"organizations:events-v2": False,
# Enable multi project selection
"organizations:global-views": False,
# Turns on grouping info.
"organizations:grouping-info": False,
# Lets organizations upgrade grouping configs and tweak them
"organizations:tweak-grouping-config": True,
# Lets organizations manage grouping configs
"organizations:set-grouping-config": False,
# Enable incidents feature
"organizations:incidents": False,
# Enable integration functionality to create and link groups to issues on
# external services.
"organizations:integrations-issue-basic": True,
# Enable interface functionality to synchronize groups between sentry and
# issues on external services.
"organizations:integrations-issue-sync": True,
# Enable interface functionality to recieve event hooks.
"organizations:integrations-event-hooks": False,
# Special feature flag primarily used on the sentry.io SAAS product for
# easily enabling features while in early development.
"organizations:internal-catchall": False,
# Enable inviting members to organizations.
"organizations:invite-members": True,
# Enable org-wide saved searches and user pinned search
"organizations:org-saved-searches": False,
# Enable the relay functionality, for use with sentry semaphore. See
# https://github.com/getsentry/semaphore.
"organizations:relay": False,
# Sentry 10 - multi project interfaces.
"organizations:sentry10": True,
# Enable basic SSO functionality, providing configurable single sign on
# using services like GitHub / Google. This is *not* the same as the signup
# and login with Github / Azure DevOps that sentry.io provides.
"organizations:sso-basic": True,
# Enable SAML2 based SSO functionality. getsentry/sentry-auth-saml2 plugin
# must be installed to use this functionality.
"organizations:sso-saml2": True,
# Enable Rippling SSO functionality.
"organizations:sso-rippling": False,
# Enable functionality to specify custom inbound filters on events.
"projects:custom-inbound-filters": False,
# Enable data forwarding functionality for projects.
"projects:data-forwarding": True,
# Enable functionality to discard groups.
"projects:discard-groups": False,
# DEPRECATED: pending removal
"projects:dsym": False,
# Enable functionality for attaching minidumps to events and displaying
# then in the group UI.
"projects:minidump": True,
# Enable functionality for project plugins.
"projects:plugins": True,
# Enable functionality for rate-limiting events on projects.
"projects:rate-limits": True,
# Enable functionality for sampling of events on projects.
"projects:sample-events": False,
# Enable functionality to trigger service hooks upon event ingestion.
"projects:servicehooks": False,
# Use Kafka (instead of Celery) for ingestion pipeline.
"projects:kafka-ingest": False,
# Don't add feature defaults down here! Please add them in their associated
# group sorted alphabetically.
}
上面是目前 Sentry 支持的特性配置，具体查看 GitHub
修改方式是在 sentry.conf.py 文件中添加配置：
SENTRY_FEATURES['projects:custom-inbound-filters'] = True # SSO ON our OFF SENTRY_FEATURES['organizations:sso'] = True SENTRY_FEATURES['organizations:sso-saml2'] = True SENTRY_FEATURES['organizations:sso-rippling'] = True
2020-5-13 周三
Pandas to_csv
使用 Pandas 的 to_csv 函数导出结果后，用 Excel 打开会乱码
加上 encoding 选项后依然没有生效
result.to_csv('result.csv', encoding="utf-8")
正确的做法是把 utf-8 改成 utf_8_sig
result.to_csv('result.csv', encoding="utf_8_sig")
有可能是版本问题，好像 pandas 1.x 版本就没有这个问题了
2020-5-14 周四
Redis 缓存
缓存穿透
这是由于进行 Redis Key 的查询， 如果不存在 Value 的话，会在数据库中查询，这个并发请求量一旦突然很大的话，会突然加重数据库的压力
解决：这种情况可能是恶意穿透，可以由前端进行一定的风险控制
缓存雪崩
当缓存服务器重新启动，或者大量的缓存都集中在同一个时间段内失效的话，那么这时候会给数据库系统带来很大的压力
解决：连接数限制，错误阈值限制，超时处理，缓存失效均匀分布，前端永不失效和后端主动更新
Elastic Docker
Elastic 是用的官方自己维护的镜像版本：
https://www.docker.elastic.co/
2020-5-15 周五
查看 Linux 的守护进程
ps -eo ppid,pid,sid,stat,tty,comm | awk '{ if ($2 == $3 && $5 == "?") {print $0}; }'
查看某个进程的状态
先可以使用 top 命令把进程们都列出来
cat /proc/{PID}/status
2020-5-18 周一
Linux 文件目录
bin 二进制可执行文件目录
boot 系统引导
dev 设备
etc 系统配置
home 用户文件的根目录
lib 程序运行所使用的共享库以及内核模块
proc 虚拟文件系统，当前内存的映射
root 超级用户目录
sbin 二进制可执行文件目录（root 用户）
tmp 临时文件
usr 系统应用程序
var 程序运行时需要改变数据的文件
2020-5-19 周二
导出 Excel 设定列宽
需要 implements WithEvents
class DataExport implements FromCollection, WithMapping, WithHeadings,
WithTitle, WithStrictNullComparison, WithEvents
{
public function __construct($start, $end, $userIds, $headers, $type)
{
// ...
}
public function collection()
{
// ...
}
public function map($row): array
{
// ...
}
public function title(): string
{
// ...
}
public function headings(): array
{
// ...
}
public function registerEvents(): array
{
return [
AfterSheet::class => function (AfterSheet $afterSheet) {
$columns = [
'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'
];
foreach ($columns as $column) {
$afterSheet->sheet->getDelegate()->getColumnDimension($column)
->setWidth(15);
}
}
];
}
}
2020-5-20 周三
Laravel 关联查询排序
可以使用 withCount 的方法
->withCount(['user_posts', 'user_address'])
->orderBy('user_posts_count', 'DESC')
2020-5-21 周四
Model 问题
Model 中的静态方法的名称不要和数据库中的字段名重复
否则在查询/取数据的时候会出现问题
2020-5-22 周五
Dingo 路由冲突
{"message":"sha1() expects parameter 1 to be string, object given",
"status_code":500,"debug":
{"line":565,
"file":"F:\\develop\\tabllapi\\vendor\\dingo\\api\\src\\Routing\\Router.php","class":"ErrorException","trace":["#0 [internal function]:
Illuminate\\Foundation\\Bootstrap\\HandleExceptions->handleError(2, 'sha1() expects ...', 'F:\\\\develop\\\\tabl...', 565, Array)","#1
...
...
路由冲突了
原先设置成了 Dingo 接管所有路由了
解决方案：把 Dingo 的路由设置和 admin 的路由设置区分开即可
2020-5-25 周一
Flask 自动重启
需要当修改了文件的时候自动重启
开启 DEBUG 模式可以自动重启：
DEBUG = True
没开 DEBUG 模式设置 RELOADER 也可以自动重启：
USE_RELOADER = True
2020-5-26 周二
单元测试报错
1) Tests\Feature\ExampleTest::testBasicTest RuntimeException: No application encryption key has been specified.
原因是单元测试连接的数据库的 APP_KEY 是用：
php artisan key:generate
随机生成的，这和原先测试数据库中的密钥不同
2020-5-27 周三
引入 dcat laravel admin 后 ci 报错
错误：
Fatal error in tests/Feature/Api/CalendarControllerTest.php:
PHP Fatal error: Uncaught ReflectionException: Class config does not exist
in /home/gitlab-runner/builds/F_P9j3J3/0/Tabll/tabllapi/vendor/laravel/framework/src/Illuminate/Container/Container.php:805
Stack trace:
#0 /home/gitlab-runner/builds/F_P9j3J3/0/Tabll/tabllapi/vendor/laravel/fram
ework/src/Illuminate/Container/Container.php(805): ReflectionClass->__construct('config')
解决方法：
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
<exclude>
<directory suffix=".php">./app/Admin</directory>
</exclude>
</whitelist>
</filter>
在 phpunit.xml 文件中加上 exclude ，排除 Admin 相关代码
2020-5-28 周四
Larave pluck 多层
使用 with 查询了两层关联的时候
->with([
'companions.user:id,name,username,mobile'
])
在 Resource 里面这样取：
'companions' => $this->companions->pluck('user.name'),
2020-5-29 周五
日期格式
有一些小区别：
Y 是年份，如 2020
y 是末位年份，如 20
m 是月份，如 02
n 是月份，如 2