发表于 | 阅读次数: 
1
2
3
4
5
6
git init
git add *
git commit -m "init"
git remote add origin  https://github.com/User/Project.git
git pull origin master --allow-unrelated-histories
git push -u origin master

发表于 | 阅读次数:

git origin是远程主机
可以通过git remove -v 看到

git push origin master的全部含义是把 本地的master分支推送到远程的master

git pull origin master的全部含义同～

发表于 | 分类于 | 阅读次数:

可持久化对象不如直接多线程dataloader。

想把处理好的对象直接保存下来，然后写了一个Cache类

第一次会把对象保存到硬盘, 第二次会直接读取

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import os
import torch
from abc import ABCMeta, abstractmethod


class DataBase(object):
    __metaclass__ = ABCMeta

    def __init__(self):
        pass

    @abstractmethod
    def get_data(self):
        pass


class Cache:
    def __init__(self, path):
        self.path = path

    def fetch(self, name, database):
        save_path = os.path.join(self.path, name)
        if os.path.exists(save_path):
            return torch.load(save_path)
        else:
            data = database.get_data()
            torch.save(data, save_path)
            return data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class TrainDataBase(DataBase):
    def get_data(self):
        return [ {'image': data['image'], 'pose': data['pose']}
                for i, data in tqdm(enumerate(train_dataloader))]


class TestDataBase(DataBase):
    def get_data(self):
        return [ {'image': data['image'], 'pose': data['pose']}
                for i, data in tqdm(enumerate(test_dataloader)) ]


persistent_path = "/SSD1/save_obj"
cache = Cache(persistent_path)
time1 = time()

train_data_vec = cache.fetch("mobile-net-train-data", TrainDataBase())
time2 = time()

test_data_vec = cache.fetch("mobile-net-test-data", TestDataBase())
time3 = time()
print ("%s, %s\n", time2 - time1, time3 - time1)

然后测试结果是
74.46734094619751s, 75.0271668434143s
本来多线程是不到1分钟。。。
去生成的文件瞅瞅，12G
因为Dataloader是多线程处理的，所以会快很多。

发表于 | 阅读次数:

tqdm 用于terminal
tqdm_notebook 用于jupyter

发表于 | 分类于 | 阅读次数:

代码: https://github.com/youngsterxyf/mpdp-code

创建型

1. 工厂模式

场景:
客户端1可以请求一个对象，而无需知道这个对象来自哪里
创建对象的代码和使用对象的代码解耦
我们执行单个函数，传入一个参数(提供信息表明我们想要什么)，但
并不要求知道任何关于对象如何实现以及对象来自哪里的细节。

阅读全文 »

发表于 | 阅读次数: 
1
2
3
4
5
6
7
8
9
%load_ext autoreload
%autoreload 2

from foo import some_function
some_function()
42
# open foo.py in an editor and change some_function to return 43
some_function()
43

发表于 | 阅读次数:

加载预训练模型

1
2
my_resnet = MyResNet(*args, **kwargs)
my_resnet.load_state_dict(torch.load("my_resnet.pth"))
1
torch.save(my_resnet.state_dict(), "my_resnet.pth")
1
my_resnet = torch.load("my_resnet.pth")

加载部分预训练模型

1
2
3
4
5
6
7
8
pretrained_dict = model_zoo.load_url(model_urls['resnet152'])
model_dict = model.state_dict()
# 将pretrained_dict里不属于model_dict的键剔除掉
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
# 更新现有的model_dict
model_dict.update(pretrained_dict)
# 加载我们真正需要的state_dict
model.load_state_dict(model_dict)

发表于 | 阅读次数: 
1
tar cf - ./images -P | pv -s $(du -sb ./images | awk '{print $1}') | gzip > /disk3/images.tar.gz

发表于 | 阅读次数:

Put your cursor on the first # character, press CtrlV (or CtrlQ for gVim), and go down until the last commented line and press x, that will delete all the # characters vertically.

For commenting a block of text is almost the same:

First, go to the first line you want to comment, press CtrlV. This will put the editor in the VISUAL BLOCK mode.
Then using the arrow key and select until the last line
Now press ShiftI, which will put the editor in INSERT mode and then press #. This will add a hash to the first line.
Then press Esc (give it a second), and it will insert a # character on all other selected lines.

https://stackoverflow.com/questions/1676632/whats-a-quick-way-to-comment-uncomment-lines-in-vim

发表于 | 阅读次数: 
1
2
pip uninstall ipykernel
pip install ipykernel